c httpclient抓取网页(【魔兽世界】《守望先锋》登录结果公布啦！ )

　　c httpclient抓取网页(【魔兽世界】《守望先锋》登录结果公布啦！

)

　　登录页面：

　　登录结果页面： , , ...

　　登录时需要用户名和密码，属性分别为tbUserName和tbPassword；如图：

　　

　　登录成功。进入下一页时需要提供cookies，也可以设置UA等属性：

　　

　　具体代码：

　　 1 package com.arlen.login;

2

3 import org.apache.commons.httpclient.Cookie;

4 import org.apache.commons.httpclient.HttpClient;

5 import org.apache.commons.httpclient.NameValuePair;

6 import org.apache.commons.httpclient.cookie.CookiePolicy;

7 import org.apache.commons.httpclient.methods.GetMethod;

8 import org.apache.commons.httpclient.methods.PostMethod;

9 import org.apache.commons.httpclient.params.HttpMethodParams;

10

11 public class ImitateLogin {

12

13 public static void main(String[] args) {

14 String userName = "username";

15 String password = "password"

16 String loginUrl = "http://passport.cnblogs.com/login.aspx";

17 String dataUrl = "http://home.cnblogs.com/";

18 HttpClientLogin(userName, password, loginUrl, dataUrl);

19 }

20

21 private static void HttpClientLogin(String userName, String password,

22 String loginUrl, String dataUrl) {

23 HttpClient httpClient = new HttpClient();

24 httpClient.getParams().setParameter(

25 HttpMethodParams.HTTP_CONTENT_CHARSET, "utf-8");

26 PostMethod postMethod = new PostMethod(loginUrl);

27

28 NameValuePair[] postData = { new NameValuePair("tbUserName", userName),

29 new NameValuePair("tbPassword", password) };

30 postMethod.setRequestBody(postData);

31

32 try {

33

34 httpClient.getParams().setCookiePolicy(

35 CookiePolicy.BROWSER_COMPATIBILITY);

36 httpClient.executeMethod(postMethod);

37 Cookie[] cookies = httpClient.getState().getCookies();

38 StringBuffer stringBuffer = new StringBuffer();

39 for (Cookie c : cookies) {

40 stringBuffer.append(c.toString() + ";");

41 }

42

43 GetMethod getMethod = new GetMethod(dataUrl);

44 getMethod.setRequestHeader("Cookie", stringBuffer.toString());

45 postMethod.setRequestHeader("Host", "passport.cnblogs.com");

46 postMethod.setRequestHeader("Referer", "http://home.cnblogs.com/");

47 postMethod.setRequestHeader("User-Agent", "AndroidCnblogs");

48 httpClient.executeMethod(getMethod);

49

50 String result = getMethod.getResponseBodyAsString();

51 System.out.println(result);

52

53 } catch (Exception e) {

54 e.printStackTrace();

55 }

56 }

57

58 }