Class HttpConnection

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String CONTENT_ENCODING  
      static java.lang.String CONTENT_TYPE  
      static java.lang.String DEFAULT_UA
      Many users would get caught by not setting a user-agent and therefore getting different responses on their desktop vs in jsoup, which would otherwise default to Java.
      static java.lang.String FORM_URL_ENCODED  
      static java.lang.String MULTIPART_FORM_DATA  
    • Constructor Summary

      Constructors 
      Constructor Description
      HttpConnection()
      Creates a new, empty HttpConnection.
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      static Connection connect​(java.lang.String url)
      Create a new Connection, with the request URL specified.
      static Connection connect​(java.net.URL url)
      Create a new Connection, with the request URL specified.
      Connection cookie​(java.lang.String name, java.lang.String value)
      Set a cookie to be sent in the request.
      Connection cookies​(java.util.Map<java.lang.String,​java.lang.String> cookies)
      Adds each of the supplied cookies to the request.
      java.net.CookieStore cookieStore()
      Get the cookie store used by this Connection.
      Connection cookieStore​(java.net.CookieStore cookieStore)
      Provide a custom or pre-filled CookieStore to be used on requests made by this Connection.
      Connection.KeyVal data​(java.lang.String key)
      Get the data KeyVal for this key, if any
      Connection data​(java.lang.String... keyvals)
      Add one or more request key, val data parameter pairs.
      Connection data​(java.lang.String key, java.lang.String value)
      Add a request data parameter.
      Connection data​(java.lang.String key, java.lang.String filename, java.io.InputStream inputStream)
      Add an input stream as a request data parameter.
      Connection data​(java.lang.String key, java.lang.String filename, java.io.InputStream inputStream, java.lang.String contentType)
      Add an input stream as a request data parameter.
      Connection data​(java.util.Collection<Connection.KeyVal> data)
      Adds all of the supplied data to the request data parameters
      Connection data​(java.util.Map<java.lang.String,​java.lang.String> data)
      Adds all of the supplied data to the request data parameters
      Connection.Response execute()
      Execute the request.
      Connection followRedirects​(boolean followRedirects)
      Configures the connection to (not) follow server redirects.
      Document get()
      Execute the request as a GET, and parse the result.
      Connection header​(java.lang.String name, java.lang.String value)
      Set a request header.
      Connection headers​(java.util.Map<java.lang.String,​java.lang.String> headers)
      Adds each of the supplied headers to the request.
      Connection ignoreContentType​(boolean ignoreContentType)
      Ignore the document's Content-Type when parsing the response.
      Connection ignoreHttpErrors​(boolean ignoreHttpErrors)
      Configures the connection to not throw exceptions when a HTTP error occurs.
      Connection maxBodySize​(int bytes)
      Set the maximum bytes to read from the (uncompressed) connection into the body, before the connection is closed, and the input truncated (i.e.
      Connection method​(Connection.Method method)
      Set the request method to use, GET or POST.
      Connection newRequest()
      Creates a new request, using this Connection as the session-state and to initialize the connection settings (which may then be independently on the returned Connection.Request object).
      Connection parser​(Parser parser)
      Provide an alternate parser to use when parsing the response to a Document.
      Document post()
      Execute the request as a POST, and parse the result.
      Connection postDataCharset​(java.lang.String charset)
      Sets the default post data character set for x-www-form-urlencoded post data
      Connection proxy​(java.lang.String host, int port)
      Set the HTTP proxy to use for this request.
      Connection proxy​(java.net.Proxy proxy)
      Set the proxy to use for this request.
      Connection referrer​(java.lang.String referrer)
      Set the request referrer (aka "referer") header.
      Connection.Request request()
      Get the request object associated with this connection
      Connection request​(Connection.Request request)
      Set the connection's request
      Connection requestBody​(java.lang.String body)
      Set a POST (or PUT) request body.
      Connection.Response response()
      Get the response, once the request has been executed.
      Connection response​(Connection.Response response)
      Set the connection's response
      Connection sslSocketFactory​(javax.net.ssl.SSLSocketFactory sslSocketFactory)
      Set custom SSL socket factory
      Connection timeout​(int millis)
      Set the total request timeout duration.
      Connection url​(java.lang.String url)
      Set the request URL to fetch.
      Connection url​(java.net.URL url)
      Set the request URL to fetch.
      Connection userAgent​(java.lang.String userAgent)
      Set the request user-agent header.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • CONTENT_ENCODING

        public static final java.lang.String CONTENT_ENCODING
        See Also:
        Constant Field Values
      • DEFAULT_UA

        public static final java.lang.String DEFAULT_UA
        Many users would get caught by not setting a user-agent and therefore getting different responses on their desktop vs in jsoup, which would otherwise default to Java. So by default, use a desktop UA.
        See Also:
        Constant Field Values
      • MULTIPART_FORM_DATA

        public static final java.lang.String MULTIPART_FORM_DATA
        See Also:
        Constant Field Values
      • FORM_URL_ENCODED

        public static final java.lang.String FORM_URL_ENCODED
        See Also:
        Constant Field Values
    • Constructor Detail

      • HttpConnection

        public HttpConnection()
        Creates a new, empty HttpConnection.
    • Method Detail

      • connect

        public static Connection connect​(java.lang.String url)
        Create a new Connection, with the request URL specified.
        Parameters:
        url - the URL to fetch from
        Returns:
        a new Connection object
      • connect

        public static Connection connect​(java.net.URL url)
        Create a new Connection, with the request URL specified.
        Parameters:
        url - the URL to fetch from
        Returns:
        a new Connection object
      • newRequest

        public Connection newRequest()
        Description copied from interface: Connection
        Creates a new request, using this Connection as the session-state and to initialize the connection settings (which may then be independently on the returned Connection.Request object).
        Specified by:
        newRequest in interface Connection
        Returns:
        a new Connection object, with a shared Cookie Store and initialized settings from this Connection and Request
      • url

        public Connection url​(java.net.URL url)
        Description copied from interface: Connection
        Set the request URL to fetch. The protocol must be HTTP or HTTPS.
        Specified by:
        url in interface Connection
        Parameters:
        url - URL to connect to
        Returns:
        this Connection, for chaining
      • url

        public Connection url​(java.lang.String url)
        Description copied from interface: Connection
        Set the request URL to fetch. The protocol must be HTTP or HTTPS.
        Specified by:
        url in interface Connection
        Parameters:
        url - URL to connect to
        Returns:
        this Connection, for chaining
      • proxy

        public Connection proxy​(@Nullable
                                java.net.Proxy proxy)
        Description copied from interface: Connection
        Set the proxy to use for this request. Set to null to disable a previously set proxy.
        Specified by:
        proxy in interface Connection
        Parameters:
        proxy - proxy to use
        Returns:
        this Connection, for chaining
      • proxy

        public Connection proxy​(java.lang.String host,
                                int port)
        Description copied from interface: Connection
        Set the HTTP proxy to use for this request.
        Specified by:
        proxy in interface Connection
        Parameters:
        host - the proxy hostname
        port - the proxy port
        Returns:
        this Connection, for chaining
      • userAgent

        public Connection userAgent​(java.lang.String userAgent)
        Description copied from interface: Connection
        Set the request user-agent header.
        Specified by:
        userAgent in interface Connection
        Parameters:
        userAgent - user-agent to use
        Returns:
        this Connection, for chaining
        See Also:
        DEFAULT_UA
      • timeout

        public Connection timeout​(int millis)
        Description copied from interface: Connection
        Set the total request timeout duration. If a timeout occurs, an SocketTimeoutException will be thrown.

        The default timeout is 30 seconds (30,000 millis). A timeout of zero is treated as an infinite timeout.

        Note that this timeout specifies the combined maximum duration of the connection time and the time to read the full response.

        Specified by:
        timeout in interface Connection
        Parameters:
        millis - number of milliseconds (thousandths of a second) before timing out connects or reads.
        Returns:
        this Connection, for chaining
        See Also:
        Connection.maxBodySize(int)
      • maxBodySize

        public Connection maxBodySize​(int bytes)
        Description copied from interface: Connection
        Set the maximum bytes to read from the (uncompressed) connection into the body, before the connection is closed, and the input truncated (i.e. the body content will be trimmed). The default maximum is 2MB. A max size of 0 is treated as an infinite amount (bounded only by your patience and the memory available on your machine).
        Specified by:
        maxBodySize in interface Connection
        Parameters:
        bytes - number of bytes to read from the input before truncating
        Returns:
        this Connection, for chaining
      • followRedirects

        public Connection followRedirects​(boolean followRedirects)
        Description copied from interface: Connection
        Configures the connection to (not) follow server redirects. By default this is true.
        Specified by:
        followRedirects in interface Connection
        Parameters:
        followRedirects - true if server redirects should be followed.
        Returns:
        this Connection, for chaining
      • referrer

        public Connection referrer​(java.lang.String referrer)
        Description copied from interface: Connection
        Set the request referrer (aka "referer") header.
        Specified by:
        referrer in interface Connection
        Parameters:
        referrer - referrer to use
        Returns:
        this Connection, for chaining
      • method

        public Connection method​(Connection.Method method)
        Description copied from interface: Connection
        Set the request method to use, GET or POST. Default is GET.
        Specified by:
        method in interface Connection
        Parameters:
        method - HTTP request method
        Returns:
        this Connection, for chaining
      • ignoreHttpErrors

        public Connection ignoreHttpErrors​(boolean ignoreHttpErrors)
        Description copied from interface: Connection
        Configures the connection to not throw exceptions when a HTTP error occurs. (4xx - 5xx, e.g. 404 or 500). By default this is false; an IOException is thrown if an error is encountered. If set to true, the response is populated with the error body, and the status message will reflect the error.
        Specified by:
        ignoreHttpErrors in interface Connection
        Parameters:
        ignoreHttpErrors - - false (default) if HTTP errors should be ignored.
        Returns:
        this Connection, for chaining
      • ignoreContentType

        public Connection ignoreContentType​(boolean ignoreContentType)
        Description copied from interface: Connection
        Ignore the document's Content-Type when parsing the response. By default this is false, an unrecognised content-type will cause an IOException to be thrown. (This is to prevent producing garbage by attempting to parse a JPEG binary image, for example.) Set to true to force a parse attempt regardless of content type.
        Specified by:
        ignoreContentType in interface Connection
        Parameters:
        ignoreContentType - set to true if you would like the content type ignored on parsing the response into a Document.
        Returns:
        this Connection, for chaining
      • data

        public Connection data​(java.lang.String key,
                               java.lang.String value)
        Description copied from interface: Connection
        Add a request data parameter. Request parameters are sent in the request query string for GETs, and in the request body for POSTs. A request may have multiple values of the same name.
        Specified by:
        data in interface Connection
        Parameters:
        key - data key
        value - data value
        Returns:
        this Connection, for chaining
      • sslSocketFactory

        public Connection sslSocketFactory​(javax.net.ssl.SSLSocketFactory sslSocketFactory)
        Description copied from interface: Connection
        Set custom SSL socket factory
        Specified by:
        sslSocketFactory in interface Connection
        Parameters:
        sslSocketFactory - custom SSL socket factory
        Returns:
        this Connection, for chaining
      • data

        public Connection data​(java.lang.String key,
                               java.lang.String filename,
                               java.io.InputStream inputStream)
        Description copied from interface: Connection
        Add an input stream as a request data parameter. For GETs, has no effect, but for POSTS this will upload the input stream.
        Specified by:
        data in interface Connection
        Parameters:
        key - data key (form item name)
        filename - the name of the file to present to the remove server. Typically just the name, not path, component.
        inputStream - the input stream to upload, that you probably obtained from a FileInputStream. You must close the InputStream in a finally block.
        Returns:
        this Connections, for chaining
        See Also:
        if you want to set the uploaded file's mimetype.
      • data

        public Connection data​(java.lang.String key,
                               java.lang.String filename,
                               java.io.InputStream inputStream,
                               java.lang.String contentType)
        Description copied from interface: Connection
        Add an input stream as a request data parameter. For GETs, has no effect, but for POSTS this will upload the input stream.
        Specified by:
        data in interface Connection
        Parameters:
        key - data key (form item name)
        filename - the name of the file to present to the remove server. Typically just the name, not path, component.
        inputStream - the input stream to upload, that you probably obtained from a FileInputStream.
        contentType - the Content Type (aka mimetype) to specify for this file. You must close the InputStream in a finally block.
        Returns:
        this Connections, for chaining
      • data

        public Connection data​(java.util.Map<java.lang.String,​java.lang.String> data)
        Description copied from interface: Connection
        Adds all of the supplied data to the request data parameters
        Specified by:
        data in interface Connection
        Parameters:
        data - map of data parameters
        Returns:
        this Connection, for chaining
      • data

        public Connection data​(java.lang.String... keyvals)
        Description copied from interface: Connection
        Add one or more request key, val data parameter pairs.

        Multiple parameters may be set at once, e.g.: .data("name", "jsoup", "language", "Java", "language", "English"); creates a query string like: ?name=jsoup&language=Java&language=English

        For GET requests, data parameters will be sent on the request query string. For POST (and other methods that contain a body), they will be sent as body form parameters, unless the body is explicitly set by Connection.requestBody(String), in which case they will be query string parameters.

        Specified by:
        data in interface Connection
        Parameters:
        keyvals - a set of key value pairs.
        Returns:
        this Connection, for chaining
      • data

        public Connection data​(java.util.Collection<Connection.KeyVal> data)
        Description copied from interface: Connection
        Adds all of the supplied data to the request data parameters
        Specified by:
        data in interface Connection
        Parameters:
        data - collection of data parameters
        Returns:
        this Connection, for chaining
      • data

        public Connection.KeyVal data​(java.lang.String key)
        Description copied from interface: Connection
        Get the data KeyVal for this key, if any
        Specified by:
        data in interface Connection
        Parameters:
        key - the data key
        Returns:
        null if not set
      • requestBody

        public Connection requestBody​(java.lang.String body)
        Description copied from interface: Connection
        Set a POST (or PUT) request body. Useful when a server expects a plain request body, not a set for URL encoded form key/value pairs. E.g.:
        Jsoup.connect(url)
         .requestBody(json)
         .header("Content-Type", "application/json")
         .post();
        If any data key/vals are supplied, they will be sent as URL query params.
        Specified by:
        requestBody in interface Connection
        Returns:
        this Request, for chaining
      • header

        public Connection header​(java.lang.String name,
                                 java.lang.String value)
        Description copied from interface: Connection
        Set a request header.
        Specified by:
        header in interface Connection
        Parameters:
        name - header name
        value - header value
        Returns:
        this Connection, for chaining
        See Also:
        Connection.Base.headers()
      • headers

        public Connection headers​(java.util.Map<java.lang.String,​java.lang.String> headers)
        Description copied from interface: Connection
        Adds each of the supplied headers to the request.
        Specified by:
        headers in interface Connection
        Parameters:
        headers - map of headers name -> value pairs
        Returns:
        this Connection, for chaining
        See Also:
        Connection.Base.headers()
      • cookie

        public Connection cookie​(java.lang.String name,
                                 java.lang.String value)
        Description copied from interface: Connection
        Set a cookie to be sent in the request.
        Specified by:
        cookie in interface Connection
        Parameters:
        name - name of cookie
        value - value of cookie
        Returns:
        this Connection, for chaining
      • cookies

        public Connection cookies​(java.util.Map<java.lang.String,​java.lang.String> cookies)
        Description copied from interface: Connection
        Adds each of the supplied cookies to the request.
        Specified by:
        cookies in interface Connection
        Parameters:
        cookies - map of cookie name -> value pairs
        Returns:
        this Connection, for chaining
      • cookieStore

        public Connection cookieStore​(java.net.CookieStore cookieStore)
        Description copied from interface: Connection
        Provide a custom or pre-filled CookieStore to be used on requests made by this Connection.
        Specified by:
        cookieStore in interface Connection
        Parameters:
        cookieStore - a cookie store to use for subsequent requests
        Returns:
        this Connection, for chaining
      • cookieStore

        public java.net.CookieStore cookieStore()
        Description copied from interface: Connection
        Get the cookie store used by this Connection.
        Specified by:
        cookieStore in interface Connection
        Returns:
        the cookie store
      • parser

        public Connection parser​(Parser parser)
        Description copied from interface: Connection
        Provide an alternate parser to use when parsing the response to a Document. If not set, defaults to the HTML parser, unless the response content-type is XML, in which case the XML parser is used.
        Specified by:
        parser in interface Connection
        Parameters:
        parser - alternate parser
        Returns:
        this Connection, for chaining
      • get

        public Document get()
                     throws java.io.IOException
        Description copied from interface: Connection
        Execute the request as a GET, and parse the result.
        Specified by:
        get in interface Connection
        Returns:
        parsed Document
        Throws:
        java.net.MalformedURLException - if the request URL is not a HTTP or HTTPS URL, or is otherwise malformed
        HttpStatusException - if the response is not OK and HTTP response errors are not ignored
        UnsupportedMimeTypeException - if the response mime type is not supported and those errors are not ignored
        java.net.SocketTimeoutException - if the connection times out
        java.io.IOException - on error
      • post

        public Document post()
                      throws java.io.IOException
        Description copied from interface: Connection
        Execute the request as a POST, and parse the result.
        Specified by:
        post in interface Connection
        Returns:
        parsed Document
        Throws:
        java.net.MalformedURLException - if the request URL is not a HTTP or HTTPS URL, or is otherwise malformed
        HttpStatusException - if the response is not OK and HTTP response errors are not ignored
        UnsupportedMimeTypeException - if the response mime type is not supported and those errors are not ignored
        java.net.SocketTimeoutException - if the connection times out
        java.io.IOException - on error
      • execute

        public Connection.Response execute()
                                    throws java.io.IOException
        Description copied from interface: Connection
        Execute the request.
        Specified by:
        execute in interface Connection
        Returns:
        a response object
        Throws:
        java.net.MalformedURLException - if the request URL is not a HTTP or HTTPS URL, or is otherwise malformed
        HttpStatusException - if the response is not OK and HTTP response errors are not ignored
        UnsupportedMimeTypeException - if the response mime type is not supported and those errors are not ignored
        java.net.SocketTimeoutException - if the connection times out
        java.io.IOException - on error
      • postDataCharset

        public Connection postDataCharset​(java.lang.String charset)
        Description copied from interface: Connection
        Sets the default post data character set for x-www-form-urlencoded post data
        Specified by:
        postDataCharset in interface Connection
        Parameters:
        charset - character set to encode post data
        Returns:
        this Connection, for chaining