HttpClient的简单封装,静态调用,自动识别网页字符集,伪装火狐/IE浏览器
HttpClient是一个非常好用的java开源项目,其作用是对用java程序对网站发起Http请求。下面是鲁炬对HttpClient进行的简单封装,主要优点是,静态调用,自动识别网页字符集,伪装火狐/IE浏览器。为什么不使用单例。以前是用单例模式,只创建一个HttpClient示例,后来发现用单例在并发情况下会出现bug,所以改为了每次调用都新建一个。import java....
·
HttpClient是一个非常好用的java开源项目,其作用是对用java程序对网站发起Http请求。
下面是鲁炬对HttpClient进行的简单封装,主要优点是,静态调用,自动识别网页字符集,伪装火狐/IE浏览器。
为什么不使用单例。以前是用单例模式,只创建一个HttpClient示例,后来发现用单例在并发情况下会出现bug,所以改为了每次调用都新建一个。
import java.io.IOException;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.lang.StringUtils;
/**
* @author 鲁炬
*
*/
public class HttpClientUtil {
public static HttpClient getClient() {
HttpClient client = new HttpClient();
return client;
}
public static String getHtml(String url) throws HttpException, IOException {
return getHtml(url, 80, null, null, 0, null);
}
public static String getHtml(String url, String cookie) throws HttpException, IOException {
return getHtml(url, 80, null, null, 0, cookie);
}
public static String getHtml(String url, int port, String cookie) throws HttpException, IOException {
return getHtml(url, port, null, null, 0, cookie);
}
public static String getHtml(String url, int port, String encoding, String proxyHost, int proxyPort, String cookie)
throws HttpException, IOException {
HttpClient httpClient = getClient();
String rest = null;
if(proxyHost != null && proxyPort != 0) httpClient.getHostConfiguration().setProxy(proxyHost, proxyPort);
HttpMethod method = new GetMethod(url);
if(!StringUtils.isBlank(cookie)) {
method.addRequestHeader("Cookie", cookie);
}
method.addRequestHeader("User-Agent", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727");
//Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
httpClient.executeMethod(method);
//根据http头解析正确的字符集
String header = method.getResponseHeader("Content-Type").getValue();
if(header.contains("charset=")) {
encoding = header.substring(header.indexOf("charset=") + "charset=".length(), header.length());
}
if(encoding == null) encoding = "GBK";
rest = new String(method.getResponseBody(), encoding);
method.releaseConnection();
return rest;
}
public static void main(String[] args) throws HttpException, IOException {
String url = "http://www.ccb.com";
System.out.println(getHtml(url));
}
}
开放原子开发者工作坊旨在鼓励更多人参与开源活动,与志同道合的开发者们相互交流开发经验、分享开发心得、获取前沿技术趋势。工作坊有多种形式的开发者活动,如meetup、训练营等,主打技术交流,干货满满,真诚地邀请各位开发者共同参与!
更多推荐
已为社区贡献7条内容
所有评论(0)