使用单线程查询10万条数据太慢了,采用多线程进行查询,提高效率。
ThredQuery查询数据的方法
public class ThredQuery implements Callable<List<HashVO>> {
private MyUtil util = new MyUtil();
private int bindex;//当前页数
private int num;//每页查询多少条
private List page;//每次分页查出来的数据
private String customerName;
private String customerType;
private String idType;
private String idNumb;
private String updateTime;
public ThredQuery(int bindex,int num,String customerName, String customerType, String idType, String idNumb, String updateTime) {
this.bindex=bindex;
this.num=num;
this.customerName=customerName;
this.customerType=customerType;
this.idType=idType;
this.idNumb=idNumb;
this.updateTime=updateTime;
}
@Override
public List<HashVO> call() throws Exception {
String sql = "";
if (StringUtils.isNotBlank(customerName)) {
sql += " and customer_name like '%" + customerName + "%' ";
}
if (StringUtils.isNotBlank(customerType)) {
sql += " and customer_type = '" + customerType + "' ";
}
if (StringUtils.isNotBlank(idType)) {
sql += " and id_type = '" + idType + "' ";
}
if (StringUtils.isNotBlank(idNumb)) {
sql += " and id_numb like '%" + idNumb + "%' ";
}
if (StringUtils.isNotBlank(updateTime)) {
sql += " and (" + updateTime + ")";
}
sql += " order by id";
String finalSql = "SELECT * FROM " +
"(" +
"SELECT A.*, ROWNUM RN " +
"FROM ("
+ sql +
" ) A " +
" WHERE ROWNUM <= "+ (bindex*num)+
" ) " +
"WHERE RN >= "+(((bindex-1)*num)+1);
return Arrays.asList(util.getHashVOs(finalSql));
}
}
注意:call里面就是查询方法。util就是查询数据的,执行sql
oracle分页方法,以pageIndex和pageSize为例
String sql= "SELECT * FROM " +
"(" +
"SELECT A.*, ROWNUM RN " +
"FROM ("
+ sql +
" ) A " +
" WHERE ROWNUM <= "+ (pageIndex*pageSize)+
" ) " +
"WHERE RN >= "+(((pageIndex-1)*pageSize)+1);
调用方法:
private void getVoListMultiThreading(String customerName, String customerType, String idType, String idNumb, String updateTime) {
long start = System.currentTimeMillis();//开始时间
List<HashVO> result = new ArrayList<>();//返回结果
//查询数据库总数量
int count = queryCount();
int num = 8000;//一次查询多少条
//需要查询的次数
int times = count / num;
if (count % num != 0) {
times = times + 1;
}
//开始页数 连接的是orcle的数据库 封装的分页方式 我的是从1开始
//Callable用于产生结果
List<Callable<List<HashVO>>> tasks = new ArrayList<>();
for (int i = 1; i <= times; i++) {
//多线程查询
Callable<List<HashVO>> qfe = new ThredQuery(i, num, customerName, customerType, idType, idNumb, updateTime);
tasks.add(qfe);
}
//定义固定长度的线程池 防止线程过多
ExecutorService executorService = Executors.newFixedThreadPool(15);
//Future用于获取结果
try {
List<Future<List<HashVO>>> futures=executorService.invokeAll(tasks);
//处理线程返回结果
if(futures!=null&&futures.size()>0){
for (Future<List<HashVO>> future:futures){
result.addAll(future.get());
}
}
} catch (InterruptedException e) {
throw new RuntimeException(e);
} catch (ExecutionException e) {
throw new RuntimeException(e);
}
executorService.shutdown();//关闭线程池
long end = System.currentTimeMillis();//结束时间
log.info("查询耗时:" + (end - start) / 1000 + "秒 ","结果: "+result);
}
查询时间仅用了30s