![]() Yikes! Using HttpClient with Partial Responses Assuming a URL requested returned an average of 10k bytes of data, that would be 1 gig of data a minute. ![]() As you might expect hitting that many URLs and retrieving the entire HTTP response, when all you need are a few bytes to verify the content would incur a tremendous amount of network traffic. I’m talking about maybe 100,000 urls that get on average checked once every minute. ![]() Here’s some background: I’m building a monitoring application that might be monitoring a huge number of URLs that get checked frequently for uptime. Why partial Requests? Why does this matter? I’ll start this post by saying I didn’t find a full solution to this problem, but I’ll layout some of the discoveries I made in my quest for small byte counts on the wire some of which partially address the issue. NET stack automatically reads a fairly large chunk of data in the first request – presumably to capture the HTTP headers. NET 4.5, or even HttpWebRequest/Response (on which the new HttpClient is based) because the. Seems easy enough, but it turns out that if you want to control bandwidth and only read a small amount of partial data from the TCP/IP connection, that process is not easy to accomplish using the new HttpClient introduced in. Over the last few days I’ve been struggling with an issue to capture HTTP content from arbitrary URLs and read only a specified number of bytes from the connection.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |