c#抓取网页数据(使用C#登陆带用户名和密码的网站,并获得网页源码 )

优采云 发布时间: 2021-11-06 08:20

  c#抓取网页数据(使用C#登陆带用户名和密码的网站,并获得网页源码

)

  诚恳请教C#登录网站获取网页和链接网页数据。使用C#以用户名密码登录网站,获取网页源代码;这个问题已经解决了!部分代码如下:

  关键是这个登录网站有很多链接,我想得到这些链接页面(即子页面)的源码,怎么做?

  感谢您的帮助!我哥有急事!

  //获取数据时调用

  privatevoidgetPage(Stringurl,StringparamList)

  {

  HttpWebRequestreq=(HttpWebRequest)WebRequest.Create(url);

  req.Method="POST";

  req.KeepAlive=true;

  req.ContentType="application/x-www-form-urlencoded";

  CookieContainercookieCon=newCookieContainer();

  req.CookieContainer=cookieCon;

  stringcookieheader=req.CookieContainer.GetCookieHeader(newUri(url));

  req.CookieContainer.SetCookies(newUri(url),cookieheader);

  byte[]SomeBytes=Encoding.UTF8.GetBytes(paramList);

  req.ContentLength=SomeBytes.Length;

  StreamnewStream=req.GetRequestStream();

  newStream.Write(SomeBytes,0,SomeBytes.Length);

  newStream.Close();

  HttpWebResponseres=(HttpWebResponse)req.GetResponse();

  StreamReceiveStream=res.GetResponseStream();

  byte[]buffer=newbyte[1024];

  stringfilename=@"c:\1\source.html";

  StreamoutStream=File.Create(filename);

  国际;

  做

  {

  l=ReceiveStream.Read(buffer,0,buffer.Length);

  if(l>0)

  outStream.Write(buffer,0,l);

  }while(l>0);

  outStream.Close();

  }

  --------------------编程问答--------------------请求子页面时,附加 Cookie

  ....

  CookieContainercookieCon=newCookieContainer();

  cookieCon.SetCookies(newUri(url),"与 Url 关联的 Cookie 字符串");

  req.CookieContainer=cookieCon;

  --------------------编程问答--------------------

  HttpWebResponse res = (HttpWebResponse)req.GetResponse(); <br />

        Stream ReceiveStream = res.GetResponseStream(); <br />

StreamReader reader=new StreamReader(ReceiveStream);//套接StreamReader <br />

String source=reader.ReadToEnd();//一次读取所有相应内容<br />

............<br />

<br />

Stream outStream = File.Create(filename); <br />

StreamWriter writer=new StreamWriter(outStream);//套接StreamWriter<br />

writer.Write(source);//将读取到的内容一次写入文件<br />

outStream.Close();<br />

writer.Close();

  这似乎比你的更有效率。

  --------------------编程问答--------------------关注。帮顶一次! --------------------编程问答--------------------叮叮更健康---- - ---------------编程问答--------------------引用3楼gisfarmer的回复:注意。一次!

  补充:.NET技术 , C#

0 个评论

要回复文章请先登录注册


官方客服QQ群

微信人工客服

QQ人工客服


线