Linux: curl (download web page)

Download One Web Page

# download a html page
curl -O http://example.org/xyz/cat.html

Download Image Sequence

# download all jpg files named cat01.jpg to cat20.jpg
curl -O http://example.org/xyz/cat[01-20].jpg

# download all jpg files named cat1.jpg to cat20.jpg
curl -O http://example.org/xyz/cat[1-20].jpg

Other Useful Options

--referer http://example.org/

Set a referer (that is, a link you came from). Some sites refuse to show content without this.

--user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:103.0) Gecko/20100101 Firefox/103.0"

Set user agent. Some sites refuse to show content without this.

• JS DOM: Get User Agent, Cookie Enabled Status

curl doc

curl -h for help.

curl -h
Usage: curl [options...] <url>
 -d, --data <data>            HTTP POST data
 -f, --fail                   Fail fast with no output on HTTP errors
 -I, --head                   Show document info only
 -H, --header <header/@file>  Pass custom header(s) to server
 -h, --help <subject>         Get help for commands
 -o, --output <file>          Write to file instead of stdout
 -O, --remote-name            Write output to file named as remote file
 -i, --show-headers           Show response headers in output
 -s, --silent                 Silent mode
 -T, --upload-file <file>     Transfer local FILE to destination
 -u, --user <user:password>   Server user and password
 -A, --user-agent <name>      Send User-Agent <name> to server
 -v, --verbose                Make the operation more talkative
 -V, --version                Show version number and quit

This is not the full help; this menu is split into categories.
Use "--help category" to get an overview of all categories, which are:
auth, connection, curl, deprecated, dns, file, ftp, global, http, imap, ldap, output,
pop3, post, proxy, scp, sftp, smtp, ssh, telnet, tftp, timeout, tls, upload, verbose.
Use "--help all" to list all options