MAJOR=4
MINOR=22
DEVNAME=tty22
     ˆÁÀƒh¹¿¾      MAJOR=7
MINOR=64
DEVNAME=vcsu
     H4030_’I|¥‰ÓMjqØ‚¦ð¬÷ÁÀ  :   ˆl–Ðz¾””†» íÆCp,©‹FÿÃ„Â¿a–ßi~”ªi?u8 ƒp/\jbÑ¿Â ?÷     # The Art Of Scripting HTTP Requests Using Curl

## Background

 This document assumes that you're familiar with HTML and general networking.

 The increasing amount of applications moving to the web has made "HTTP
 Scripting" more frequently requested and wanted. To be able to automatically
 extract information from the web, to fake users, to post or upload data to
 web servers are all important tasks today.

 Curl is a command line tool for doing all sorts of URL manipulations and
 transfers, but this particular document will focus on how to use it when
 doing HTTP requests for fun and profit. I will assume that you know how to
 invoke `curl --help` or `curl --manual` to get basic information about it.

 Curl is not written to do everything for you. It makes the requests, it gets
 the data, it sends data and it retrieves the information. You probably need
 to glue everything together using some kind of script language or repeated
 manual invokes.

## The HTTP Protocol

 HTTP is the protocol used to fetch data from web servers. It is a very simple
 protocol that is built upon TCP/IP. The protocol also allows information to
 get sent to the server from the client using a few different methods, as will
 be shown here.

 HTTP is plain ASCII text lines being sent by the client to a server to
 request a particular action, and then the server replies a few text lines
 before the actual requested content is sent to the client.

 The client, curl, sends a HTTP request. The request contains a method (like
 GET, POST, HEAD etc), a number of request headers and sometimes a request
 body. The HTTP server responds with a status line (indicating if things went
 well), response headers and most often also a response body. The "body" part
 is the plain data you requested, like the actual HTML or the image etc.

## See the Protocol

 Using curl's option [`--verbose`](https://curl.se/docs/manpage.html#-v)
 (`-v` as a short option) will display what kind of commands curl sends to the
 server, as well as a few other informational texts.

 `--verbose` is the single most useful option when it comes to debug or even
 understand the curl<->server interaction.

 Sometimes even `--verbose` is not enough. Then
 [`--trace`](https://curl.se/docs/manpage.html#-trace) and
 [`--trace-ascii`]((https://curl.se/docs/manpage.html#--trace-ascii)
 offer even more details as they show **everything** curl sends and
 receives. Use it like this:

    curl --trace-ascii debugdump.txt http://www.example.com/

## See the Timing

 Many times you may wonder what exactly is taking all the time, or you just
 want to know the amount of milliseconds between two points in a transfer. For
 those, and other similar situations, the
 [`--trace-time`]((https://curl.se/docs/manpage.html#--trace-time) option
 is what you need. It'll prepend the time to each trace output line:

    curl --trace-ascii d.txt --trace-time http://example.com/

## See the Response

 By default curl sends the response to stdout. You need to redirect it
 somewhere to avoid that, most often that is done with ` -o` or `-O`.

# URL

## Spec

 The Uniform Resource Locator format is how you specify the address of a
 particular resource on the Internet. You know these, you've seen URLs like
 https://curl.se or https://yourbank.com a million times. RFC 3986 is the
 canonical spec. And yeah, the formal name is not URL, it is URI.

## Host

 The host name is usually resolved using DNS or your /etc/hosts file to an IP
 address and that's what curl will communicate with. Alternatively you specify
 the IP address directly in the URL instead of a name.

 For development and other trying out situations, you can point to a different
 IP address for a host name than what would otherwise be used, by using curl's
 [`--resolve`](https://curl.se/docs/manpage.html#--resolve) option:

    curl --resolve www.example.org:80