java爬虫抓取网页数据( post方式就要考虑提交的表单内容怎么传输了呢？ )

　　java爬虫抓取网页数据(

post方式就要考虑提交的表单内容怎么传输了呢？

)

　　public static String httpPostWithJSON(String url) throws Exception {

HttpPost httpPost = new HttpPost(url);

CloseableHttpClient client = HttpClients.createDefault();

String respContent = null;

// json方式

JSONObject jsonParam = new JSONObject();

jsonParam.put("name", "admin");

jsonParam.put("pass", "123456");

StringEntity entity = new StringEntity(jsonParam.toString(),"utf-8");//解决中文乱码问题

entity.setContentEncoding("UTF-8");

entity.setContentType("application/json");

httpPost.setEntity(entity);

System.out.println();

// 表单方式

// List pairList = new ArrayList();

// pairList.add(new BasicNameValuePair("name", "admin"));

// pairList.add(new BasicNameValuePair("pass", "123456"));

// httpPost.setEntity(new UrlEncodedFormEntity(pairList, "utf-8"));

HttpResponse resp = client.execute(httpPost);

if(resp.getStatusLine().getStatusCode() == 200) {

HttpEntity he = resp.getEntity();

respContent = EntityUtils.toString(he,"UTF-8");

}

return respContent;

}

public static void main(String[] args) throws Exception {

String result = httpPostWithJSON("http://localhost:8080/hcTest2/Hc");

System.out.println(result);

}

　　post方法需要考虑提交的表单内容是如何传输的。在本文中，name 和 pass 是表单的值。

　　封装形式的属性可以使用json或者传统形式。如果是传统形式，请注意，即注释上面的代码部分。这样，在servlet中，也就是数据处理层可以直接通过request.getParameter("string")获取属性值。它比json简单，但是在实际开发中，一般都是用json来进行数据传输。使用json有两种选择，一种是阿里巴巴的fastjson，一种是谷歌的gson。相比fastjson，效率更高，gson适合解析常规JSON数据。博主在这里使用fastjson。另外，如果使用json，在数据处理层需要使用streams来读取表单属性，比传统表单内容要多一点。代码已经存在了。

　　public class HcServlet extends HttpServlet {

private static final long serialVersionUID = 1L;

protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {

doPost(request, response);

}

protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {

request.setCharacterEncoding("UTF-8");

response.setContentType("text/html;charset=UTF-8");

String acceptjson = "";

User user = new User();

BufferedReader br = new BufferedReader(new InputStreamReader(

(ServletInputStream) request.getInputStream(), "utf-8"));

StringBuffer sb = new StringBuffer("");

String temp;

while ((temp = br.readLine()) != null) {

sb.append(temp);

}

br.close();

acceptjson = sb.toString();

if (acceptjson != "") {

JSONObject jo = JSONObject.parseObject(acceptjson);

user.setUsername(jo.getString("name"));

user.setPassword(jo.getString("pass"));

}

request.setAttribute("user", user);

request.getRequestDispatcher("/message.jsp").forward(request, response);

}

}