HTTPClient

The HTTPClient class provides several methods for accessing Web resources via HTTP.

HTTPClient instance is designed to be MT-safe. You can call a HTTPClient instance from several threads without synchronization after setting up an instance.

  clnt = HTTPClient.new
  clnt.set_cookie_store('/home/nahi/cookie.dat')
  urls.each do |url|
    Thread.new(url) do |u|
      p clnt.head(u).status
    end
  end

How to use

At first, how to create your client. See initialize for more detail.

Create simple client.
```
 clnt = HTTPClient.new
```
Accessing resources through HTTP proxy. You can use environment variable ‘http_proxy’ or ‘HTTP_PROXY’ instead.
```
 clnt = HTTPClient.new('http://myproxy:8080')
```

How to retrieve web resources

See get_content.

Get content of specified URL. It returns a String of whole result.
```
 puts clnt.get_content('http://dev.ctor.org/')
```

Get content as chunks of String. It yields chunks of String.

 clnt.get_content('http://dev.ctor.org/') do |chunk|
   puts chunk
 end

Invoking other HTTP methods

See head, get, post, put, delete, options, propfind, proppatch and trace. It returns a HTTP::Message instance as a response.

Do HEAD request.

 res = clnt.head(uri)
 p res.header['Last-Modified'][0]

Do GET request with query.

 query = { 'keyword' => 'ruby', 'lang' => 'en' }
 res = clnt.get(uri, query)
 p res.status
 p res.contenttype
 p res.header['X-Custom']
 puts res.content

How to POST

See post.

Do POST a form data.

 body = { 'keyword' => 'ruby', 'lang' => 'en' }
 res = clnt.post(uri, body)

Do multipart file upload with POST. No need to set extra header by yourself from httpclient/2.1.4.

 File.open('/tmp/post_data') do |file|
   body = { 'upload' => file, 'user' => 'nahi' }
   res = clnt.post(uri, body)
 end

Accessing via SSL

Ruby needs to be compiled with OpenSSL.

Get content of specified URL via SSL. Just pass an URL which starts with ‘https://’.
```
 https_url = 'https://www.rsa.com'
 clnt.get_content(https_url)
```

Getting peer certificate from response.

 res = clnt.get(https_url)
 p res.peer_cert #=> returns OpenSSL::X509::Certificate

Configuring OpenSSL options. See HTTPClient::SSLConfig for more details.

 user_cert_file = 'cert.pem'
 user_key_file = 'privkey.pem'
 clnt.ssl_config.set_client_cert_file(user_cert_file, user_key_file)
 clnt.get_content(https_url)

Handling Cookies

Using volatile Cookies. Nothing to do. HTTPClient handles Cookies.

 clnt = HTTPClient.new
 clnt.get_content(url1) # receives Cookies.
 clnt.get_content(url2) # sends Cookies if needed.

Saving non volatile Cookies to a specified file. Need to set a file at first and invoke save method at last.

 clnt = HTTPClient.new
 clnt.set_cookie_store('/home/nahi/cookie.dat')
 clnt.get_content(url)
 ...
 clnt.save_cookie_store

Disabling Cookies.

 clnt = HTTPClient.new
 clnt.cookie_manager = nil

Configuring authentication credentials

Authentication with Web server. Supports BasicAuth, DigestAuth, and Negotiate/NTLM (requires ruby/ntlm module).

 clnt = HTTPClient.new
 domain = 'http://dev.ctor.org/http-access2/'
 user = 'user'
 password = 'user'
 clnt.set_auth(domain, user, password)
 p clnt.get_content('http://dev.ctor.org/http-access2/login').status

Authentication with Proxy server. Supports BasicAuth and NTLM (requires win32/sspi)

 clnt = HTTPClient.new(proxy)
 user = 'proxy'
 password = 'proxy'
 clnt.set_proxy_auth(user, password)
 p clnt.get_content(url)

Invoking HTTP methods with custom header

Pass a Hash or an Array for extheader argument.

    extheader = { 'Accept' => '*/*' }
    clnt.get_content(uri, query, extheader)

    extheader = [['Accept', 'image/jpeg'], ['Accept', 'image/png']]
    clnt.get_content(uri, query, extheader)

Invoking HTTP methods asynchronously

See head_async, get_async, post_async, put_async, delete_async, options_async, propfind_async, proppatch_async, and trace_async. It immediately returns a HTTPClient::Connection instance as a returning value.

    connection = clnt.post_async(url, body)
    print 'posting.'
    while true
      break if connection.finished?
      print '.'
      sleep 1
    end
    puts '.'
    res = connection.pop
    p res.status
    p res.content.read # res.content is an IO for the res of async method.

Shortcut methods

You can invoke get_content, get, etc. without creating HTTPClient instance.

  ruby -rhttpclient -e 'puts HTTPClient.get_content(ARGV.shift)' http://dev.ctor.org/
  ruby -rhttpclient -e 'p HTTPClient.head(ARGV.shift).header["last-modified"]' http://dev.ctor.org/

This program is copyrighted free software by NAKAMURA, Hiroshi. You can redistribute it and/or modify it under the same terms of Ruby’s license; either the dual license version in 2003, or any later version.

Constants

VERSION
RUBY_VERSION_STRING
LIB_NAME
PROPFIND_DEFAULT_EXTHEADER: Default extheader for PROPFIND request.
NO_PROXY_HOSTS
NTLMEnabled
NTLMEnabled
SSPIEnabled
SSPIEnabled
SSLEnabled
SSLEnabled

Attributes

ssl_config[R]

HTTPClient::SSLConfig

SSL configurator.

test_loopback_response[R]

An array of response HTTP message body String which is used for loop-back test. See test/* to see how to use it. If you want to do loop-back test of HTTP header, use test_loopback_http_response instead.

request_filter[R]

An array of request filter which can trap HTTP request/response. See HTTPClient::WWWAuth to see how to use it.

proxy_auth[R]

HTTPClient::ProxyAuth

Proxy authentication handler.

www_auth[R]

HTTPClient::WWWAuth

WWW authentication handler.

follow_redirect_count[RW]

How many times get_content and post_content follows HTTP redirect. 10 by default.

Public Class Methods

new(*args) click to toggle source

Creates a HTTPClient instance which manages sessions, cookies, etc.

HTTPClient.new takes 3 optional arguments for proxy url string, User-Agent String and From header String. User-Agent and From are embedded in HTTP request Header if given. No User-Agent and From header added without setting it explicitly.

  proxy = 'http://myproxy:8080'
  agent_name = 'MyAgent/0.1'
  from = 'from@example.com'
  HTTPClient.new(proxy, agent_name, from)

You can use a keyword argument style Hash. Keys are :proxy, :agent_name and :from.

  HTTPClient.new(:agent_name = 'MyAgent/0.1')

     # File lib/httpclient.rb, line 340
340:   def initialize(*args)
341:     proxy, agent_name, from = keyword_argument(args, :proxy, :agent_name, :from)
342:     @proxy = nil        # assigned later.
343:     @no_proxy = nil
344:     @www_auth = WWWAuth.new
345:     @proxy_auth = ProxyAuth.new
346:     @request_filter = [@proxy_auth, @www_auth]
347:     @debug_dev = nil
348:     @redirect_uri_callback = method(:default_redirect_uri_callback)
349:     @test_loopback_response = []
350:     @session_manager = SessionManager.new(self)
351:     @session_manager.agent_name = agent_name
352:     @session_manager.from = from
353:     @session_manager.ssl_config = @ssl_config = SSLConfig.new(self)
354:     @cookie_manager = WebAgent::CookieManager.new
355:     @follow_redirect_count = 10
356:     load_environment
357:     self.proxy = proxy if proxy
358:   end

timeout_scheduler() click to toggle source

CAUTION: caller must aware of race condition.

     # File lib/httpclient/timeout.rb, line 115
115:     def timeout_scheduler
116:       @timeout_scheduler ||= TimeoutScheduler.new
117:     end

Private Class Methods

attr_proxy(symbol, assignable = false) click to toggle source

     # File lib/httpclient.rb, line 266
266:     def attr_proxy(symbol, assignable = false)
267:       name = symbol.to_s
268:       define_method(name) {
269:         @session_manager.__send__(name)
270:       }
271:       if assignable
272:         aname = name + '='
273:         define_method(aname) { |rhs|
274:           reset_all
275:           @session_manager.__send__(aname, rhs)
276:         }
277:       end
278:     end

Public Instance Methods

debug_dev() click to toggle source

Returns debug device if exists. See debug_dev=.

     # File lib/httpclient.rb, line 361
361:   def debug_dev
362:     @debug_dev
363:   end

debug_dev=(dev) click to toggle source

Sets debug device. Once debug device is set, all HTTP requests and responses are dumped to given device. dev must respond to << for dump.

Calling this method resets all existing sessions.

     # File lib/httpclient.rb, line 369
369:   def debug_dev=(dev)
370:     @debug_dev = dev
371:     reset_all
372:     @session_manager.debug_dev = dev
373:   end

default_redirect_uri_callback(uri, res) click to toggle source

A default method for redirect uri callback. This method is used by HTTPClient instance by default. This callback allows relative redirect such as

  Location: ../foo/

in HTTP header.

     # File lib/httpclient.rb, line 570
570:   def default_redirect_uri_callback(uri, res)
571:     newuri = URI.parse(res.header['location'][0])
572:     if https?(uri) && !https?(newuri)
573:       raise BadResponseError.new("redirecting to non-https resource")
574:     end
575:     unless newuri.is_a?(URI::HTTP)
576:       newuri = uri + newuri
577:       STDERR.puts("could be a relative URI in location header which is not recommended")
578:       STDERR.puts("'The field value consists of a single absolute URI' in HTTP spec")
579:     end
580:     puts "redirect to: #{newuri}" if $DEBUG
581:     newuri
582:   end

delete(uri, extheader = {}, &block) click to toggle source

Sends DELETE request to the specified URL. See request for arguments.

     # File lib/httpclient.rb, line 605
605:   def delete(uri, extheader = {}, &block)
606:     request(:delete, uri, nil, nil, extheader, &block)
607:   end

delete_async(uri, extheader = {}) click to toggle source

Sends DELETE request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.

     # File lib/httpclient.rb, line 695
695:   def delete_async(uri, extheader = {})
696:     request_async(:delete, uri, nil, nil, extheader)
697:   end

get(uri, query = nil, extheader = {}, &block) click to toggle source

Sends GET request to the specified URL. See request for arguments.

     # File lib/httpclient.rb, line 590
590:   def get(uri, query = nil, extheader = {}, &block)
591:     request(:get, uri, query, nil, extheader, &block)
592:   end

get_async(uri, query = nil, extheader = {}) click to toggle source

Sends GET request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.

     # File lib/httpclient.rb, line 677
677:   def get_async(uri, query = nil, extheader = {})
678:     request_async(:get, uri, query, nil, extheader)
679:   end

get_content(uri, query = nil, extheader = {}, &block) click to toggle source

Retrieves a web resource.

uri	a String or an URI object which represents an URL of web resource.
query	a Hash or an Array of query part of URL. e.g. { “a” => “b” } => ‘host/part?a=b’. Give an array to pass multiple value like [“a”, “b”], [“a”, “c”] > ‘host/part?a=b&a=c’.
extheader	a Hash or an Array of extra headers. e.g. { ‘Accept’ => ‘/’ } or [[‘Accept’, ‘image/jpeg’], [‘Accept’, ‘image/png’]].
&block	Give a block to get chunked message-body of response like get_content(uri) { \|chunked_body\| … }. Size of each chunk may not be the same.

get_content follows HTTP redirect status (see HTTP::Status.redirect?) internally and try to retrieve content from redirected URL. See redirect_uri_callback= how HTTP redirection is handled.

If you need to get full HTTP response including HTTP status and headers, use get method. get returns HTTP::Message as a response and you need to follow HTTP redirect by yourself if you need.

     # File lib/httpclient.rb, line 518
518:   def get_content(uri, query = nil, extheader = {}, &block)
519:     follow_redirect(:get, uri, query, nil, extheader, &block).content
520:   end

head(uri, query = nil, extheader = {}) click to toggle source

Sends HEAD request to the specified URL. See request for arguments.

     # File lib/httpclient.rb, line 585
585:   def head(uri, query = nil, extheader = {})
586:     request(:head, uri, query, nil, extheader)
587:   end

head_async(uri, query = nil, extheader = {}) click to toggle source

Sends HEAD request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.

     # File lib/httpclient.rb, line 671
671:   def head_async(uri, query = nil, extheader = {})
672:     request_async(:head, uri, query, nil, extheader)
673:   end

no_proxy() click to toggle source

Returns NO_PROXY setting String if given.

     # File lib/httpclient.rb, line 411
411:   def no_proxy
412:     @no_proxy
413:   end

no_proxy=(no_proxy) click to toggle source

Sets NO_PROXY setting String. no_proxy must be a comma separated String. Each entry must be ‘host’ or ‘host:port’ such as; HTTPClient#no_proxy = ‘example.com,example.co.jp:443’

‘localhost’ is treated as a no_proxy site regardless of explicitly listed. HTTPClient checks given URI objects before accessing it. ‘host’ is tail string match. No IP-addr conversion.

You can use environment variable ‘no_proxy’ or ‘NO_PROXY’ for it.

Calling this method resets all existing sessions.

     # File lib/httpclient.rb, line 426
426:   def no_proxy=(no_proxy)
427:     @no_proxy = no_proxy
428:     reset_all
429:   end

options(uri, extheader = {}, &block) click to toggle source

Sends OPTIONS request to the specified URL. See request for arguments.

     # File lib/httpclient.rb, line 610
610:   def options(uri, extheader = {}, &block)
611:     request(:options, uri, nil, nil, extheader, &block)
612:   end

options_async(uri, extheader = {}) click to toggle source

Sends OPTIONS request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.

     # File lib/httpclient.rb, line 701
701:   def options_async(uri, extheader = {})
702:     request_async(:options, uri, nil, nil, extheader)
703:   end

post(uri, body = '', extheader = {}, &block) click to toggle source

Sends POST request to the specified URL. See request for arguments.

     # File lib/httpclient.rb, line 595
595:   def post(uri, body = '', extheader = {}, &block)
596:     request(:post, uri, nil, body, extheader, &block)
597:   end

post_async(uri, body = nil, extheader = {}) click to toggle source

Sends POST request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.

     # File lib/httpclient.rb, line 683
683:   def post_async(uri, body = nil, extheader = {})
684:     request_async(:post, uri, nil, body, extheader)
685:   end

post_content(uri, body = nil, extheader = {}, &block) click to toggle source

Posts a content.

uri	a String or an URI object which represents an URL of web resource.
body	a Hash or an Array of body part. e.g. { “a” => “b” } => ‘a=b’. Give an array to pass multiple value like [“a”, “b”], [“a”, “c”] > ‘a=b&a=c’. When you pass a File as a value, it will be posted as a multipart/form-data. e.g. { ‘upload’ => file }
extheader	a Hash or an Array of extra headers. e.g. { ‘Accept’ => ‘/’ } or [[‘Accept’, ‘image/jpeg’], [‘Accept’, ‘image/png’]].
&block	Give a block to get chunked message-body of response like post_content(uri) { \|chunked_body\| … }. Size of each chunk may not be the same.

post_content follows HTTP redirect status (see HTTP::Status.redirect?) internally and try to post the content to redirected URL. See redirect_uri_callback= how HTTP redirection is handled.

If you need to get full HTTP response including HTTP status and headers, use post method.

     # File lib/httpclient.rb, line 544
544:   def post_content(uri, body = nil, extheader = {}, &block)
545:     follow_redirect(:post, uri, nil, body, extheader, &block).content
546:   end

propfind(uri, extheader = PROPFIND_DEFAULT_EXTHEADER, &block) click to toggle source

Sends PROPFIND request to the specified URL. See request for arguments.

     # File lib/httpclient.rb, line 615
615:   def propfind(uri, extheader = PROPFIND_DEFAULT_EXTHEADER, &block)
616:     request(:propfind, uri, nil, nil, extheader, &block)
617:   end

propfind_async(uri, extheader = PROPFIND_DEFAULT_EXTHEADER) click to toggle source

Sends PROPFIND request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.

     # File lib/httpclient.rb, line 707
707:   def propfind_async(uri, extheader = PROPFIND_DEFAULT_EXTHEADER)
708:     request_async(:propfind, uri, nil, nil, extheader)
709:   end

proppatch(uri, body = nil, extheader = {}, &block) click to toggle source

Sends PROPPATCH request to the specified URL. See request for arguments.

     # File lib/httpclient.rb, line 620
620:   def proppatch(uri, body = nil, extheader = {}, &block)
621:     request(:proppatch, uri, nil, body, extheader, &block)
622:   end

proppatch_async(uri, body = nil, extheader = {}) click to toggle source

Sends PROPPATCH request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.

     # File lib/httpclient.rb, line 713
713:   def proppatch_async(uri, body = nil, extheader = {})
714:     request_async(:proppatch, uri, nil, body, extheader)
715:   end

proxy() click to toggle source

Returns URI object of HTTP proxy if exists.

     # File lib/httpclient.rb, line 376
376:   def proxy
377:     @proxy
378:   end

proxy=(proxy) click to toggle source

Sets HTTP proxy used for HTTP connection. Given proxy can be an URI, a String or nil. You can set user/password for proxy authentication like HTTPClient#proxy = ‘user:passwd@myproxy:8080‘

You can use environment variable ‘http_proxy’ or ‘HTTP_PROXY’ for it. You need to use ‘cgi_http_proxy’ or ‘CGI_HTTP_PROXY’ instead if you run HTTPClient from CGI environment from security reason. (HTTPClient checks ‘REQUEST_METHOD’ environment variable whether it’s CGI or not)

Calling this method resets all existing sessions.

     # File lib/httpclient.rb, line 390
390:   def proxy=(proxy)
391:     if proxy.nil?
392:       @proxy = nil
393:       @proxy_auth.reset_challenge
394:     else
395:       @proxy = urify(proxy)
396:       if @proxy.scheme == nil or @proxy.scheme.downcase != 'http' or
397:           @proxy.host == nil or @proxy.port == nil
398:         raise ArgumentError.new("unsupported proxy #{proxy}")
399:       end
400:       @proxy_auth.reset_challenge
401:       if @proxy.user || @proxy.password
402:         @proxy_auth.set_auth(@proxy.user, @proxy.password)
403:       end
404:     end
405:     reset_all
406:     @session_manager.proxy = @proxy
407:     @proxy
408:   end

put(uri, body = '', extheader = {}, &block) click to toggle source

Sends PUT request to the specified URL. See request for arguments.

     # File lib/httpclient.rb, line 600
600:   def put(uri, body = '', extheader = {}, &block)
601:     request(:put, uri, nil, body, extheader, &block)
602:   end

put_async(uri, body = nil, extheader = {}) click to toggle source

Sends PUT request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.

     # File lib/httpclient.rb, line 689
689:   def put_async(uri, body = nil, extheader = {})
690:     request_async(:put, uri, nil, body, extheader)
691:   end

redirect_uri_callback=(redirect_uri_callback) click to toggle source

Sets callback proc when HTTP redirect status is returned for get_content and post_content. default_redirect_uri_callback is used by default.

If you need strict implementation which does not allow relative URI redirection, set strict_redirect_uri_callback instead.

  clnt.redirect_uri_callback = clnt.method(:strict_redirect_uri_callback)

     # File lib/httpclient.rb, line 493
493:   def redirect_uri_callback=(redirect_uri_callback)
494:     @redirect_uri_callback = redirect_uri_callback
495:   end

request(method, uri, query = nil, body = nil, extheader = {}, &block) click to toggle source

Sends a request to the specified URL.

method	HTTP method to be sent. method.to_s.upcase is used.
uri	a String or an URI object which represents an URL of web resource.
query	a Hash or an Array of query part of URL. e.g. { “a” => “b” } => ‘host/part?a=b’ Give an array to pass multiple value like [“a”, “b”], [“a”, “c”] > ‘host/part?a=b&a=c‘
body	a Hash or an Array of body part. e.g. { “a” => “b” } => ‘a=b’. Give an array to pass multiple value like [“a”, “b”], [“a”, “c”] > ‘a=b&a=c’. When the given method is ‘POST’ and the given body contains a file as a value, it will be posted as a multipart/form-data. e.g. { ‘upload’ => file } See HTTP::Message.file? for actual condition of ‘a file’.
extheader	a Hash or an Array of extra headers. e.g. { ‘Accept’ => ‘/’ } or [[‘Accept’, ‘image/jpeg’], [‘Accept’, ‘image/png’]].
&block	Give a block to get chunked message-body of response like get(uri) { \|chunked_body\| … }. Size of each chunk may not be the same.

You can also pass a String as a body. HTTPClient just sends a String as a HTTP request message body.

When you pass an IO as a body, HTTPClient sends it as a HTTP request with chunked encoding (Transfer-Encoding: chunked in HTTP header). Bear in mind that some server application does not support chunked request. At least cgi.rb does not support it.

     # File lib/httpclient.rb, line 659
659:   def request(method, uri, query = nil, body = nil, extheader = {}, &block)
660:     uri = urify(uri)
661:     if block
662:       filtered_block = proc { |res, str|
663:         block.call(str)
664:       }
665:     end
666:     do_request(method, uri, query, body, extheader, &filtered_block)
667:   end

request_async(method, uri, query = nil, body = nil, extheader = {}) click to toggle source

Sends a request in async style. request method creates new Thread for HTTP connection and returns a HTTPClient::Connection instance immediately.

Arguments definition is the same as request.

     # File lib/httpclient.rb, line 727
727:   def request_async(method, uri, query = nil, body = nil, extheader = {})
728:     uri = urify(uri)
729:     do_request_async(method, uri, query, body, extheader)
730:   end

reset(uri) click to toggle source

Resets internal session for the given URL. Keep-alive connection for the site (host-port pair) is disconnected if exists.

     # File lib/httpclient.rb, line 734
734:   def reset(uri)
735:     uri = urify(uri)
736:     @session_manager.reset(uri)
737:   end

reset_all() click to toggle source

Resets all of internal sessions. Keep-alive connections are disconnected.

     # File lib/httpclient.rb, line 740
740:   def reset_all
741:     @session_manager.reset_all
742:   end

set_auth(domain, user, passwd) click to toggle source

Sets credential for Web server authentication.

domain

a String or an URI to specify where HTTPClient should use this

      credential.  If you set uri to nil, HTTPClient uses this credential
      wherever a server requires it.

user	username String.
passwd	password String.

You can set multiple credentials for each uri.

  clnt.set_auth('http://www.example.com/foo/', 'foo_user', 'passwd')
  clnt.set_auth('http://www.example.com/bar/', 'bar_user', 'passwd')

Calling this method resets all existing sessions.

     # File lib/httpclient.rb, line 444
444:   def set_auth(domain, user, passwd)
445:     uri = urify(domain)
446:     @www_auth.set_auth(uri, user, passwd)
447:     reset_all
448:   end

set_basic_auth(domain, user, passwd) click to toggle source

Deprecated. Use set_auth instead.

     # File lib/httpclient.rb, line 451
451:   def set_basic_auth(domain, user, passwd)
452:     uri = urify(domain)
453:     @www_auth.basic_auth.set(uri, user, passwd)
454:     reset_all
455:   end

set_proxy_auth(user, passwd) click to toggle source

Sets credential for Proxy authentication.

user	username String.
passwd	password String.

Calling this method resets all existing sessions.

     # File lib/httpclient.rb, line 462
462:   def set_proxy_auth(user, passwd)
463:     @proxy_auth.set_auth(user, passwd)
464:     reset_all
465:   end

strict_redirect_uri_callback(uri, res) click to toggle source

A method for redirect uri callback. How to use:

  clnt.redirect_uri_callback = clnt.method(:strict_redirect_uri_callback)

This callback does not allow relative redirect such as

  Location: ../foo/

in HTTP header. (raises BadResponseError instead)

     # File lib/httpclient.rb, line 553
553:   def strict_redirect_uri_callback(uri, res)
554:     newuri = URI.parse(res.header['location'][0])
555:     if https?(uri) && !https?(newuri)
556:       raise BadResponseError.new("redirecting to non-https resource")
557:     end
558:     unless newuri.is_a?(URI::HTTP)
559:       raise BadResponseError.new("unexpected location: #{newuri}", res)
560:     end
561:     puts "redirect to: #{newuri}" if $DEBUG
562:     newuri
563:   end

trace(uri, query = nil, body = nil, extheader = {}, &block) click to toggle source

Sends TRACE request to the specified URL. See request for arguments.

     # File lib/httpclient.rb, line 625
625:   def trace(uri, query = nil, body = nil, extheader = {}, &block)
626:     request('TRACE', uri, query, body, extheader, &block)
627:   end

trace_async(uri, query = nil, body = nil, extheader = {}) click to toggle source

Sends TRACE request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.

     # File lib/httpclient.rb, line 719
719:   def trace_async(uri, query = nil, body = nil, extheader = {})
720:     request_async(:trace, uri, query, body, extheader)
721:   end

Private Instance Methods

create_boundary() click to toggle source

     # File lib/httpclient.rb, line 896
896:   def create_boundary
897:     Digest::SHA1.hexdigest(Time.now.to_s)
898:   end

create_request(method, uri, query, body, extheader) click to toggle source

     # File lib/httpclient.rb, line 854
854:   def create_request(method, uri, query, body, extheader)
855:     method = method.to_s.upcase
856:     if extheader.is_a?(Hash)
857:       extheader = extheader.to_a
858:     else
859:       extheader = extheader.dup
860:     end
861:     boundary = nil
862:     if body
863:       dummy, content_type = extheader.find { |key, value|
864:         key.downcase == 'content-type'
865:       }
866:       if content_type
867:         if /\Amultipart/ =~ content_type
868:           if content_type =~ /boundary=(.+)\z/
869:             boundary = $1
870:           else
871:             boundary = create_boundary
872:             content_type = "#{content_type}; boundary=#{boundary}"
873:             extheader = override_header(extheader, 'Content-Type', content_type)
874:           end
875:         end
876:       elsif method == 'POST'
877:         if file_in_form_data?(body)
878:           boundary = create_boundary
879:           content_type = "multipart/form-data; boundary=#{boundary}"
880:         else
881:           content_type = 'application/x-www-form-urlencoded'
882:         end
883:         extheader << ['Content-Type', content_type]
884:       end
885:     end
886:     req = HTTP::Message.new_request(method, uri, query, body, boundary)
887:     extheader.each do |key, value|
888:       req.header.add(key, value)
889:     end
890:     if @cookie_manager && cookie = @cookie_manager.find(uri)
891:       req.header.add('Cookie', cookie)
892:     end
893:     req
894:   end

do_get_block(req, proxy, conn, &block) click to toggle source

!! CAUTION !!

  Method 'do_get*' runs under MT conditon. Be careful to change.

     # File lib/httpclient.rb, line 941
941:   def do_get_block(req, proxy, conn, &block)
942:     @request_filter.each do |filter|
943:       filter.filter_request(req)
944:     end
945:     if str = @test_loopback_response.shift
946:       dump_dummy_request_response(req.body.dump, str) if @debug_dev
947:       conn.push(HTTP::Message.new_response(str))
948:       return
949:     end
950:     content = block ? nil : ''
951:     res = HTTP::Message.new_response(content)
952:     @debug_dev << "= Request\n\n" if @debug_dev
953:     sess = @session_manager.query(req, proxy)
954:     res.peer_cert = sess.ssl_peer_cert
955:     @debug_dev << "\n\n= Response\n\n" if @debug_dev
956:     do_get_header(req, res, sess)
957:     conn.push(res)
958:     sess.get_body do |part|
959:       if block
960:         block.call(res, part)
961:       else
962:         content << part
963:       end
964:     end
965:     @session_manager.keep(sess) unless sess.closed?
966:     commands = @request_filter.collect { |filter|
967:       filter.filter_response(req, res)
968:     }
969:     if commands.find { |command| command == :retry }
970:       raise RetryableResponse.new
971:     end
972:   end

do_get_header(req, res, sess) click to toggle source

      # File lib/httpclient.rb, line 1002
1002:   def do_get_header(req, res, sess)
1003:     res.version, res.status, res.reason, headers = sess.get_header
1004:     headers.each do |key, value|
1005:       res.header.add(key, value)
1006:     end
1007:     if @cookie_manager
1008:       res.header['set-cookie'].each do |cookie|
1009:         @cookie_manager.parse(cookie, req.header.request_uri)
1010:       end
1011:     end
1012:   end

do_get_stream(req, proxy, conn) click to toggle source

      # File lib/httpclient.rb, line 974
 974:   def do_get_stream(req, proxy, conn)
 975:     @request_filter.each do |filter|
 976:       filter.filter_request(req)
 977:     end
 978:     if str = @test_loopback_response.shift
 979:       dump_dummy_request_response(req.body.dump, str) if @debug_dev
 980:       conn.push(HTTP::Message.new_response(StringIO.new(str)))
 981:       return
 982:     end
 983:     piper, pipew = IO.pipe
 984:     res = HTTP::Message.new_response(piper)
 985:     @debug_dev << "= Request\n\n" if @debug_dev
 986:     sess = @session_manager.query(req, proxy)
 987:     res.peer_cert = sess.ssl_peer_cert
 988:     @debug_dev << "\n\n= Response\n\n" if @debug_dev
 989:     do_get_header(req, res, sess)
 990:     conn.push(res)
 991:     sess.get_body do |part|
 992:       pipew.syswrite(part)
 993:     end
 994:     pipew.close
 995:     @session_manager.keep(sess) unless sess.closed?
 996:     commands = @request_filter.collect { |filter|
 997:       filter.filter_response(req, res)
 998:     }
 999:     # ignore commands (not retryable in async mode)
1000:   end

do_request(method, uri, query, body, extheader, &block) click to toggle source

     # File lib/httpclient.rb, line 752
752:   def do_request(method, uri, query, body, extheader, &block)
753:     conn = Connection.new
754:     res = nil
755:     if HTTP::Message.file?(body)
756:       pos = body.pos rescue nil
757:     end
758:     retry_count = @session_manager.protocol_retry_count
759:     proxy = no_proxy?(uri) ? nil : @proxy
760:     while retry_count > 0
761:       body.pos = pos if pos
762:       req = create_request(method, uri, query, body, extheader)
763:       begin
764:         protect_keep_alive_disconnected do
765:           do_get_block(req, proxy, conn, &block)
766:         end
767:         res = conn.pop
768:         break
769:       rescue RetryableResponse
770:         res = conn.pop
771:         retry_count -= 1
772:       end
773:     end
774:     res
775:   end

do_request_async(method, uri, query, body, extheader) click to toggle source

     # File lib/httpclient.rb, line 777
777:   def do_request_async(method, uri, query, body, extheader)
778:     conn = Connection.new
779:     t = Thread.new(conn) { |tconn|
780:       if HTTP::Message.file?(body)
781:         pos = body.pos rescue nil
782:       end
783:       retry_count = @session_manager.protocol_retry_count
784:       proxy = no_proxy?(uri) ? nil : @proxy
785:       while retry_count > 0
786:         body.pos = pos if pos
787:         req = create_request(method, uri, query, body, extheader)
788:         begin
789:           protect_keep_alive_disconnected do
790:             do_get_stream(req, proxy, tconn)
791:           end
792:           break
793:         rescue RetryableResponse
794:           retry_count -= 1
795:         end
796:       end
797:     }
798:     conn.async_thread = t
799:     conn
800:   end

dump_dummy_request_response(req, res) click to toggle source

      # File lib/httpclient.rb, line 1014
1014:   def dump_dummy_request_response(req, res)
1015:     @debug_dev << "= Dummy Request\n\n"
1016:     @debug_dev << req
1017:     @debug_dev << "\n\n= Dummy Response\n\n"
1018:     @debug_dev << res
1019:   end

file_in_form_data?(body) click to toggle source

     # File lib/httpclient.rb, line 900
900:   def file_in_form_data?(body)
901:     HTTP::Message.multiparam_query?(body) &&
902:       body.any? { |k, v| HTTP::Message.file?(v) }
903:   end

follow_redirect(method, uri, query, body, extheader, &block) click to toggle source

     # File lib/httpclient.rb, line 820
820:   def follow_redirect(method, uri, query, body, extheader, &block)
821:     uri = urify(uri)
822:     if block
823:       filtered_block = proc { |r, str|
824:         block.call(str) if HTTP::Status.successful?(r.status)
825:       }
826:     end
827:     if HTTP::Message.file?(body)
828:       pos = body.pos rescue nil
829:     end
830:     retry_number = 0
831:     while retry_number < @follow_redirect_count
832:       body.pos = pos if pos
833:       res = do_request(method, uri, query, body, extheader, &filtered_block)
834:       if HTTP::Status.successful?(res.status)
835:         return res
836:       elsif HTTP::Status.redirect?(res.status)
837:         uri = urify(@redirect_uri_callback.call(uri, res))
838:         retry_number += 1
839:       else
840:         raise BadResponseError.new("unexpected response: #{res.header.inspect}", res)
841:       end
842:     end
843:     raise BadResponseError.new("retry count exceeded", res)
844:   end

getenv(name) click to toggle source

     # File lib/httpclient.rb, line 816
816:   def getenv(name)
817:     ENV[name.downcase] || ENV[name.upcase]
818:   end

https?(uri) click to toggle source

     # File lib/httpclient.rb, line 935
935:   def https?(uri)
936:     uri.scheme.downcase == 'https'
937:   end

load_environment() click to toggle source

     # File lib/httpclient.rb, line 802
802:   def load_environment
803:     # http_proxy
804:     if getenv('REQUEST_METHOD')
805:       # HTTP_PROXY conflicts with the environment variable usage in CGI where
806:       # HTTP_* is used for HTTP header information.  Unlike open-uri, we
807:       # simply ignore http_proxy in CGI env and use cgi_http_proxy instead.
808:       self.proxy = getenv('cgi_http_proxy')
809:     else
810:       self.proxy = getenv('http_proxy')
811:     end
812:     # no_proxy
813:     self.no_proxy = getenv('no_proxy')
814:   end

no_proxy?(uri) click to toggle source

     # File lib/httpclient.rb, line 919
919:   def no_proxy?(uri)
920:     if !@proxy or NO_PROXY_HOSTS.include?(uri.host)
921:       return true
922:     end
923:     unless @no_proxy
924:       return false
925:     end
926:     @no_proxy.scan(/([^:,]+)(?::(\d+))?/) do |host, port|
927:       if /(\A|\.)#{Regexp.quote(host)}\z/ =~ uri.host &&
928:           (!port || uri.port == port.to_i)
929:         return true
930:       end
931:     end
932:     false
933:   end

override_header(extheader, key, value) click to toggle source

     # File lib/httpclient.rb, line 905
905:   def override_header(extheader, key, value)
906:     result = []
907:     extheader.each do |k, v|
908:       if k.downcase == key.downcase
909:         result << [key, value]
910:       else
911:         result << [k, v]
912:       end
913:     end
914:     result
915:   end

protect_keep_alive_disconnected() click to toggle source

     # File lib/httpclient.rb, line 846
846:   def protect_keep_alive_disconnected
847:     begin
848:       yield
849:     rescue KeepAliveDisconnected
850:       yield
851:     end
852:   end

Home Classes Methods

In Files

Parent

Namespace

Methods

Included Modules

Files

Class Index

HTTPClient

How to use

How to retrieve web resources

Invoking other HTTP methods

How to POST

Accessing via SSL

Handling Cookies

Configuring authentication credentials

Invoking HTTP methods with custom header

Invoking HTTP methods asynchronously

Shortcut methods

Constants

Attributes

Public Class Methods

Private Class Methods

Public Instance Methods

> ‘host/part?a=b&a=c’.

> ‘a=b&a=c’.

> ‘host/part?a=b&a=c‘

> ‘a=b&a=c’.

Private Instance Methods