12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544 |
- _ _ ____ _
- ___| | | | _ \| |
- / __| | | | |_) | |
- | (__| |_| | _ <| |___
- \___|\___/|_| \_\_____|
- FAQ
- 1. Philosophy
- 1.1 What is cURL?
- 1.2 What is libcurl?
- 1.3 What is curl not?
- 1.4 When will you make curl do XXXX ?
- 1.5 Who makes curl?
- 1.6 What do you get for making curl?
- 1.7 What about CURL from curl.com?
- 1.8 I have a problem, who do I mail?
- 1.9 Where do I buy commercial support for curl?
- 1.10 How many are using curl?
- 1.11 Why do you not update ca-bundle.crt
- 1.12 I have a problem, who can I chat with?
- 1.13 curl's ECCN number?
- 1.14 How do I submit my patch?
- 1.15 How do I port libcurl to my OS?
- 2. Install Related Problems
- 2.1 configure fails when using static libraries
- 2.2 Does curl work/build with other SSL libraries?
- 2.4 Does curl support SOCKS (RFC 1928) ?
- 3. Usage Problems
- 3.1 curl: (1) SSL is disabled, https: not supported
- 3.2 How do I tell curl to resume a transfer?
- 3.3 Why does my posting using -F not work?
- 3.4 How do I tell curl to run custom FTP commands?
- 3.5 How can I disable the Accept: */* header?
- 3.6 Does curl support ASP, XML, XHTML or HTML version Y?
- 3.7 Can I use curl to delete/rename a file through FTP?
- 3.8 How do I tell curl to follow HTTP redirects?
- 3.9 How do I use curl in my favorite programming language?
- 3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP?
- 3.11 How do I POST with a different Content-Type?
- 3.12 Why do FTP-specific features over HTTP proxy fail?
- 3.13 Why do my single/double quotes fail?
- 3.14 Does curl support JavaScript or PAC (automated proxy config)?
- 3.15 Can I do recursive fetches with curl?
- 3.16 What certificates do I need when I use SSL?
- 3.17 How do I list the root directory of an FTP server?
- 3.18 Can I use curl to send a POST/PUT and not wait for a response?
- 3.19 How do I get HTTP from a host using a specific IP address?
- 3.20 How to SFTP from my user's home directory?
- 3.21 Protocol xxx not supported or disabled in libcurl
- 3.22 curl -X gives me HTTP problems
- 4. Running Problems
- 4.2 Why do I get problems when I use & or % in the URL?
- 4.3 How can I use {, }, [ or ] to specify multiple URLs?
- 4.4 Why do I get downloaded data even though the web page does not exist?
- 4.5 Why do I get return code XXX from an HTTP server?
- 4.5.1 "400 Bad Request"
- 4.5.2 "401 Unauthorized"
- 4.5.3 "403 Forbidden"
- 4.5.4 "404 Not Found"
- 4.5.5 "405 Method Not Allowed"
- 4.5.6 "301 Moved Permanently"
- 4.6 Can you tell me what error code 142 means?
- 4.7 How do I keep user names and passwords secret in curl command lines?
- 4.8 I found a bug
- 4.9 curl cannot authenticate to a server that requires NTLM?
- 4.10 My HTTP request using HEAD, PUT or DELETE does not work
- 4.11 Why do my HTTP range requests return the full document?
- 4.12 Why do I get "certificate verify failed" ?
- 4.13 Why is curl -R on Windows one hour off?
- 4.14 Redirects work in browser but not with curl
- 4.15 FTPS does not work
- 4.16 My HTTP POST or PUT requests are slow
- 4.17 Non-functional connect timeouts on Windows
- 4.18 file:// URLs containing drive letters (Windows, NetWare)
- 4.19 Why does not curl return an error when the network cable is unplugged?
- 4.20 curl does not return error for HTTP non-200 responses
- 5. libcurl Issues
- 5.1 Is libcurl thread-safe?
- 5.2 How can I receive all data into a large memory chunk?
- 5.3 How do I fetch multiple files with libcurl?
- 5.4 Does libcurl do Winsock initialization on win32 systems?
- 5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ?
- 5.6 What about Keep-Alive or persistent connections?
- 5.7 Link errors when building libcurl on Windows
- 5.8 libcurl.so.X: open failed: No such file or directory
- 5.9 How does libcurl resolve host names?
- 5.10 How do I prevent libcurl from writing the response to stdout?
- 5.11 How do I make libcurl not receive the whole HTTP response?
- 5.12 Can I make libcurl fake or hide my real IP address?
- 5.13 How do I stop an ongoing transfer?
- 5.14 Using C++ non-static functions for callbacks?
- 5.15 How do I get an FTP directory listing?
- 5.16 I want a different time-out
- 5.17 Can I write a server with libcurl?
- 5.18 Does libcurl use threads?
- 6. License Issues
- 6.1 I have a GPL program, can I use the libcurl library?
- 6.2 I have a closed-source program, can I use the libcurl library?
- 6.3 I have a BSD licensed program, can I use the libcurl library?
- 6.4 I have a program that uses LGPL libraries, can I use libcurl?
- 6.5 Can I modify curl/libcurl for my program and keep the changes secret?
- 6.6 Can you please change the curl/libcurl license to XXXX?
- 6.7 What are my obligations when using libcurl in my commercial apps?
- 7. PHP/CURL Issues
- 7.1 What is PHP/CURL?
- 7.2 Who wrote PHP/CURL?
- 7.3 Can I perform multiple requests using the same handle?
- 7.4 Does PHP/CURL have dependencies?
- 8. Development
- 8.1 Why does curl use C89?
- 8.2 Will curl be rewritten?
- ==============================================================================
- 1. Philosophy
- 1.1 What is cURL?
- cURL is the name of the project. The name is a play on 'Client for URLs',
- originally with URL spelled in uppercase to make it obvious it deals with
- URLs. The fact it can also be read as 'see URL' also helped, it works as
- an abbreviation for "Client URL Request Library" or why not the recursive
- version: "curl URL Request Library".
- The cURL project produces two products:
- libcurl
- A client-side URL transfer library, supporting DICT, FILE, FTP, FTPS,
- GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S,
- RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS
- and WSS.
- libcurl supports HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading,
- Kerberos, SPNEGO, HTTP form based upload, proxies, cookies, user+password
- authentication, file transfer resume, http proxy tunneling and more.
- libcurl is highly portable, it builds and works identically on numerous
- platforms, including Solaris, NetBSD, FreeBSD, OpenBSD, Darwin, HP-UX,
- IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows, Amiga, OS/2, macOS,
- Ultrix, QNX, OpenVMS, RISC OS, Novell NetWare, DOS, Symbian, OSF, Android,
- Minix, IBM TPF and more...
- libcurl is free, thread-safe, IPv6 compatible, feature rich, well
- supported and fast.
- curl
- A command line tool for getting or sending data using URL syntax.
- Since curl uses libcurl, curl supports the same wide range of common
- Internet protocols that libcurl does.
- We pronounce curl with an initial k sound. It rhymes with words like girl
- and earl. This is a short WAV file to help you:
- https://media.merriam-webster.com/soundc11/c/curl0001.wav
- There are numerous sub-projects and related projects that also use the word
- curl in the project names in various combinations, but you should take
- notice that this FAQ is directed at the command-line tool named curl (and
- libcurl the library), and may therefore not be valid for other curl-related
- projects. (There is however a small section for the PHP/CURL in this FAQ.)
- 1.2 What is libcurl?
- libcurl is a reliable and portable library for doing Internet data transfers
- using one or more of its supported Internet protocols.
- You can use libcurl freely in your application, be it open source,
- commercial or closed-source.
- libcurl is most probably the most portable, most powerful and most often
- used C-based multi-platform file transfer library on this planet - be it
- open source or commercial.
- 1.3 What is curl not?
- curl is not a wget clone. That is a common misconception. Never, during
- curl's development, have we intended curl to replace wget or compete on its
- market. curl is targeted at single-shot file transfers.
- curl is not a website mirroring program. If you want to use curl to mirror
- something: fine, go ahead and write a script that wraps around curl or use
- libcurl to make it reality.
- curl is not an FTP site mirroring program. Sure, get and send FTP with curl
- but if you want systematic and sequential behavior you should write a
- script (or write a new program that interfaces libcurl) and do it.
- curl is not a PHP tool, even though it works perfectly well when used from
- or with PHP (when using the PHP/CURL module).
- curl is not a program for a single operating system. curl exists, compiles,
- builds and runs under a wide range of operating systems, including all
- modern Unixes (and a bunch of older ones too), Windows, Amiga, OS/2, macOS,
- QNX etc.
- 1.4 When will you make curl do XXXX ?
- We love suggestions of what to change in order to make curl and libcurl
- better. We do however believe in a few rules when it comes to the future of
- curl:
- curl -- the command line tool -- is to remain a non-graphical command line
- tool. If you want GUIs or fancy scripting capabilities, you should look for
- another tool that uses libcurl.
- We do not add things to curl that other small and available tools already do
- well at the side. curl's output can be piped into another program or
- redirected to another file for the next program to interpret.
- We focus on protocol related issues and improvements. If you want to do more
- magic with the supported protocols than curl currently does, chances are
- good we will agree. If you want to add more protocols, we may agree.
- If you want someone else to do all the work while you wait for us to
- implement it for you, that is not a friendly attitude. We spend a
- considerable time already on maintaining and developing curl. In order to
- get more out of us, you should consider trading in some of your time and
- effort in return. Simply go to the GitHub repository which resides at
- https://github.com/curl/curl, fork the project, and create pull requests
- with your proposed changes.
- If you write the code, chances are better that it will get into curl faster.
- 1.5 Who makes curl?
- curl and libcurl are not made by any single individual. Daniel Stenberg is
- project leader and main developer, but other persons' submissions are
- important and crucial. Anyone can contribute and post their changes and
- improvements and have them inserted in the main sources (of course on the
- condition that developers agree that the fixes are good).
- The full list of all contributors is found in the docs/THANKS file.
- curl is developed by a community, with Daniel at the wheel.
- 1.6 What do you get for making curl?
- Project cURL is entirely free and open. We do this voluntarily, mostly in
- our spare time. Companies may pay individual developers to work on curl.
- This is not controlled by nor supervised in any way by the curl project.
- We get help from companies. Haxx provides website, bandwidth, mailing lists
- etc, GitHub hosts the primary git repository and other services like the bug
- tracker at https://github.com/curl/curl. Also again, some companies have
- sponsored certain parts of the development in the past and I hope some will
- continue to do so in the future.
- If you want to support our project, consider a donation or a banner-program
- or even better: by helping us with coding, documenting or testing etc.
- See also: https://curl.se/sponsors.html
- 1.7 What about CURL from curl.com?
- During the summer of 2001, curl.com was busy advertising their client-side
- programming language for the web, named CURL.
- We are in no way associated with curl.com or their CURL programming
- language.
- Our project name curl has been in effective use since 1998. We were not the
- first computer related project to use the name "curl" and do not claim any
- rights to the name.
- We recognize that we will be living in parallel with curl.com and wish them
- every success.
- 1.8 I have a problem, who do I mail?
- Please do not mail any single individual unless you really need to. Keep
- curl-related questions on a suitable mailing list. All available mailing
- lists are listed in the MANUAL document and online at
- https://curl.se/mail/
- Keeping curl-related questions and discussions on mailing lists allows
- others to join in and help, to share their ideas, to contribute their
- suggestions and to spread their wisdom. Keeping discussions on public mailing
- lists also allows for others to learn from this (both current and future
- users thanks to the web based archives of the mailing lists), thus saving us
- from having to repeat ourselves even more. Thanks for respecting this.
- If you have found or simply suspect a security problem in curl or libcurl,
- submit all the details at https://hackerone.one/curl. On there we keep the
- issue private while we investigate, confirm it, work and validate a fix and
- agree on a time schedule for publication etc. That way we produce a fix in a
- timely manner before the flaw is announced to the world, reducing the impact
- the problem risks having on existing users.
- Security issues can also be taking to the curl security team by emailing
- security at curl.se (closed list of receivers, mails are not disclosed).
- 1.9 Where do I buy commercial support for curl?
- curl is fully open source. It means you can hire any skilled engineer to fix
- your curl-related problems.
- We list available alternatives on the curl website:
- https://curl.se/support.html
- 1.10 How many are using curl?
- It is impossible to tell.
- We do not know how many users that knowingly have installed and use curl.
- We do not know how many users that use curl without knowing that they are in
- fact using it.
- We do not know how many users that downloaded or installed curl and then
- never use it.
- In 2020, we estimate that curl runs in roughly ten billion installations
- world wide.
- 1.11 Why do you not update ca-bundle.crt
- In the cURL project we have decided not to attempt to keep this file updated
- (or even present) since deciding what to add to a ca cert bundle is an
- undertaking we have not been ready to accept, and the one we can get from
- Mozilla is perfectly fine so there is no need to duplicate that work.
- Today, with many services performed over HTTPS, every operating system
- should come with a default ca cert bundle that can be deemed somewhat
- trustworthy and that collection (if reasonably updated) should be deemed to
- be a lot better than a private curl version.
- If you want the most recent collection of ca certs that Mozilla Firefox
- uses, we recommend that you extract the collection yourself from Mozilla
- Firefox (by running 'make ca-bundle), or by using our online service setup
- for this purpose: https://curl.se/docs/caextract.html
- 1.12 I have a problem who, can I chat with?
- There is a bunch of friendly people hanging out in the #curl channel on the
- IRC network libera.chat. If you are polite and nice, chances are good that
- you can get -- or provide -- help instantly.
- 1.13 curl's ECCN number?
- The US government restricts exports of software that contains or uses
- cryptography. When doing so, the Export Control Classification Number (ECCN)
- is used to identify the level of export control etc.
- Apache Software Foundation gives a good explanation of ECCNs at
- https://www.apache.org/dev/crypto.html
- We believe curl's number might be ECCN 5D002, another possibility is
- 5D992. It seems necessary to write them (the authority that administers ECCN
- numbers), asking to confirm.
- Comprehensible explanations of the meaning of such numbers and how to obtain
- them (resp.) are here
- https://www.bis.doc.gov/licensing/exportingbasics.htm
- https://www.bis.doc.gov/licensing/do_i_needaneccn.html
- An incomprehensible description of the two numbers above is here
- https://www.bis.doc.gov/index.php/documents/new-encryption/1653-ccl5-pt2-3
- 1.14 How do I submit my patch?
- We strongly encourage you to submit changes and improvements directly as
- "pull requests" on GitHub: https://github.com/curl/curl/pulls
- If you for any reason cannot or will not deal with GitHub, send your patch to
- the curl-library mailing list. We are many subscribers there and there are
- lots of people who can review patches, comment on them and "receive" them
- properly.
- Lots of more details are found in the CONTRIBUTE.md and INTERNALS.md
- documents.
- 1.15 How do I port libcurl to my OS?
- Here's a rough step-by-step:
- 1. copy a suitable lib/config-*.h file as a start to lib/config-[youros].h
- 2. edit lib/config-[youros].h to match your OS and setup
- 3. edit lib/curl_setup.h to include config-[youros].h when your OS is
- detected by the preprocessor, in the style others already exist
- 4. compile lib/*.c and make them into a library
- 2. Install Related Problems
- 2.1 configure fails when using static libraries
- You may find that configure fails to properly detect the entire dependency
- chain of libraries when you provide static versions of the libraries that
- configure checks for.
- The reason why static libraries is much harder to deal with is that for them
- we do not get any help but the script itself must know or check what more
- libraries that are needed (with shared libraries, that dependency "chain" is
- handled automatically). This is a error-prone process and one that also
- tends to vary over time depending on the release versions of the involved
- components and may also differ between operating systems.
- For that reason, configure does few attempts to actually figure this out and
- you are instead encouraged to set LIBS and LDFLAGS accordingly when you
- invoke configure, and point out the needed libraries and set the necessary
- flags yourself.
- 2.2 Does curl work with other SSL libraries?
- curl has been written to use a generic SSL function layer internally, and
- that SSL functionality can then be provided by one out of many different SSL
- backends.
- curl can be built to use one of the following SSL alternatives: OpenSSL,
- libressl, BoringSSL, AWS-LC, GnuTLS, wolfSSL, mbedTLS, Secure Transport
- (native iOS/OS X), Schannel (native Windows), GSKit (native IBM i), BearSSL,
- or Rustls. They all have their pros and cons, and we try to maintain a
- comparison of them here: https://curl.se/docs/ssl-compared.html
- 2.4 Does curl support SOCKS (RFC 1928) ?
- Yes, SOCKS 4 and 5 are supported.
- 3. Usage problems
- 3.1 curl: (1) SSL is disabled, https: not supported
- If you get this output when trying to get anything from an https:// server,
- it means that the instance of curl/libcurl that you are using was built
- without support for this protocol.
- This could have happened if the configure script that was run at build time
- could not find all libs and include files curl requires for SSL to work. If
- the configure script fails to find them, curl is simply built without SSL
- support.
- To get the https:// support into a curl that was previously built but that
- reports that https:// is not supported, you should dig through the document
- and logs and check out why the configure script does not find the SSL libs
- and/or include files.
- Also, check out the other paragraph in this FAQ labeled "configure does not
- find OpenSSL even when it is installed".
- 3.2 How do I tell curl to resume a transfer?
- curl supports resumed transfers both ways on both FTP and HTTP.
- Try the -C option.
- 3.3 Why does my posting using -F not work?
- You cannot arbitrarily use -F or -d, the choice between -F or -d depends on
- the HTTP operation you need curl to do and what the web server that will
- receive your post expects.
- If the form you are trying to submit uses the type 'multipart/form-data',
- then and only then you must use the -F type. In all the most common cases,
- you should use -d which then causes a posting with the type
- 'application/x-www-form-urlencoded'.
- This is described in some detail in the MANUAL and TheArtOfHttpScripting
- documents, and if you do not understand it the first time, read it again
- before you post questions about this to the mailing list. Also, try reading
- through the mailing list archives for old postings and questions regarding
- this.
- 3.4 How do I tell curl to run custom FTP commands?
- You can tell curl to perform optional commands both before and/or after a
- file transfer. Study the -Q/--quote option.
- Since curl is used for file transfers, you do not normally use curl to
- perform FTP commands without transferring anything. Therefore you must
- always specify a URL to transfer to/from even when doing custom FTP
- commands, or use -I which implies the "no body" option sent to libcurl.
- 3.5 How can I disable the Accept: */* header?
- You can change all internally generated headers by adding a replacement with
- the -H/--header option. By adding a header with empty contents you safely
- disable that one. Use -H "Accept:" to disable that specific header.
- 3.6 Does curl support ASP, XML, XHTML or HTML version Y?
- To curl, all contents are alike. It does not matter how the page was
- generated. It may be ASP, PHP, Perl, shell-script, SSI or plain HTML
- files. There is no difference to curl and it does not even know what kind of
- language that generated the page.
- See also item 3.14 regarding JavaScript.
- 3.7 Can I use curl to delete/rename a file through FTP?
- Yes. You specify custom FTP commands with -Q/--quote.
- One example would be to delete a file after you have downloaded it:
- curl -O ftp://download.com/coolfile -Q '-DELE coolfile'
- or rename a file after upload:
- curl -T infile ftp://upload.com/dir/ -Q "-RNFR infile" -Q "-RNTO newname"
- 3.8 How do I tell curl to follow HTTP redirects?
- curl does not follow so-called redirects by default. The Location: header
- that informs the client about this is only interpreted if you are using the
- -L/--location option. As in:
- curl -L http://redirector.com
- Not all redirects are HTTP ones, see 4.14
- 3.9 How do I use curl in my favorite programming language?
- Many programming languages have interfaces/bindings that allow you to use
- curl without having to use the command line tool. If you are fluent in such
- a language, you may prefer to use one of these interfaces instead.
- Find out more about which languages that support curl directly, and how to
- install and use them, in the libcurl section of the curl website:
- https://curl.se/libcurl/
- All the various bindings to libcurl are made by other projects and people,
- outside of the cURL project. The cURL project itself only produces libcurl
- with its plain C API. If you do not find anywhere else to ask you can ask
- about bindings on the curl-library list too, but be prepared that people on
- that list may not know anything about bindings.
- In December 2021, there were interfaces available for the following
- languages: Ada95, Basic, C, C++, Ch, Cocoa, D, Delphi, Dylan, Eiffel,
- Euphoria, Falcon, Ferite, Gambas, glib/GTK+, Go, Guile, Harbour, Haskell,
- Java, Julia, Lisp, Lua, Mono, .NET, node.js, Object-Pascal, OCaml, Pascal,
- Perl, PHP, PostgreSQL, Python, R, Rexx, Ring, RPG, Ruby, Rust, Scheme,
- Scilab, S-Lang, Smalltalk, SP-Forth, SPL, Tcl, Visual Basic, Visual FoxPro,
- Q, wxwidgets, XBLite and Xoho. By the time you read this, additional ones
- may have appeared.
- 3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP?
- curl adheres to the HTTP spec, which basically means you can play with *any*
- protocol that is built on top of HTTP. Protocols such as SOAP, WEBDAV and
- XML-RPC are all such ones. You can use -X to set custom requests and -H to
- set custom headers (or replace internally generated ones).
- Using libcurl is of course just as good and you would just use the proper
- library options to do the same.
- 3.11 How do I POST with a different Content-Type?
- You can always replace the internally generated headers with -H/--header.
- To make a simple HTTP POST with text/xml as content-type, do something like:
- curl -d "datatopost" -H "Content-Type: text/xml" [URL]
- 3.12 Why do FTP-specific features over HTTP proxy fail?
- Because when you use an HTTP proxy, the protocol spoken on the network will
- be HTTP, even if you specify an FTP URL. This effectively means that you
- normally cannot use FTP-specific features such as FTP upload and FTP quote
- etc.
- There is one exception to this rule, and that is if you can "tunnel through"
- the given HTTP proxy. Proxy tunneling is enabled with a special option (-p)
- and is generally not available as proxy admins usually disable tunneling to
- ports other than 443 (which is used for HTTPS access through proxies).
- 3.13 Why do my single/double quotes fail?
- To specify a command line option that includes spaces, you might need to
- put the entire option within quotes. Like in:
- curl -d " with spaces " url.com
- or perhaps
- curl -d ' with spaces ' url.com
- Exactly what kind of quotes and how to do this is entirely up to the shell
- or command line interpreter that you are using. For most unix shells, you
- can more or less pick either single (') or double (") quotes. For
- Windows/DOS command prompts you must use double (") quotes, and if the
- option string contains inner double quotes you can escape them with a
- backslash.
- For Windows powershell the arguments are not always passed on as expected
- because curl is not a powershell script. You may or may not be able to use
- single quotes. To escape inner double quotes seems to require a
- backslash-backtick escape sequence and the outer quotes as double quotes.
- Please study the documentation for your particular environment. Examples in
- the curl docs will use a mix of both of these as shown above. You must
- adjust them to work in your environment.
- Remember that curl works and runs on more operating systems than most single
- individuals have ever tried.
- 3.14 Does curl support JavaScript or PAC (automated proxy config)?
- Many web pages do magic stuff using embedded JavaScript. curl and libcurl
- have no built-in support for that, so it will be treated just like any other
- contents.
- .pac files are a Netscape invention and are sometimes used by organizations
- to allow them to differentiate which proxies to use. The .pac contents is
- just a JavaScript program that gets invoked by the browser and that returns
- the name of the proxy to connect to. Since curl does not support JavaScript,
- it cannot support .pac proxy configuration either.
- Some workarounds usually suggested to overcome this JavaScript dependency:
- Depending on the JavaScript complexity, write up a script that translates it
- to another language and execute that.
- Read the JavaScript code and rewrite the same logic in another language.
- Implement a JavaScript interpreter, people have successfully used the
- Mozilla JavaScript engine in the past.
- Ask your admins to stop this, for a static proxy setup or similar.
- 3.15 Can I do recursive fetches with curl?
- No. curl itself has no code that performs recursive operations, such as
- those performed by wget and similar tools.
- There exists wrapper scripts with that functionality (for example the
- curlmirror perl script), and you can write programs based on libcurl to do
- it, but the command line tool curl itself cannot.
- 3.16 What certificates do I need when I use SSL?
- There are three different kinds of "certificates" to keep track of when we
- talk about using SSL-based protocols (HTTPS or FTPS) using curl or libcurl.
- CLIENT CERTIFICATE
- The server you communicate with may require that you can provide this in
- order to prove that you actually are who you claim to be. If the server
- does not require this, you do not need a client certificate.
- A client certificate is always used together with a private key, and the
- private key has a pass phrase that protects it.
- SERVER CERTIFICATE
- The server you communicate with has a server certificate. You can and should
- verify this certificate to make sure that you are truly talking to the real
- server and not a server impersonating it.
- CERTIFICATE AUTHORITY CERTIFICATE ("CA cert")
- You often have several CA certs in a CA cert bundle that can be used to
- verify a server certificate that was signed by one of the authorities in the
- bundle. curl does not come with a CA cert bundle but most curl installs
- provide one. You can also override the default.
- The server certificate verification process is made by using a Certificate
- Authority certificate ("CA cert") that was used to sign the server
- certificate. Server certificate verification is enabled by default in curl
- and libcurl and is often the reason for problems as explained in FAQ entry
- 4.12 and the SSLCERTS document
- (https://curl.se/docs/sslcerts.html). Server certificates that are
- "self-signed" or otherwise signed by a CA that you do not have a CA cert
- for, cannot be verified. If the verification during a connect fails, you are
- refused access. You then need to explicitly disable the verification to
- connect to the server.
- 3.17 How do I list the root directory of an FTP server?
- There are two ways. The way defined in the RFC is to use an encoded slash
- in the first path part. List the "/tmp" directory like this:
- curl ftp://ftp.sunet.se/%2ftmp/
- or the not-quite-kosher-but-more-readable way, by simply starting the path
- section of the URL with a slash:
- curl ftp://ftp.sunet.se//tmp/
- 3.18 Can I use curl to send a POST/PUT and not wait for a response?
- No.
- You can easily write your own program using libcurl to do such stunts.
- 3.19 How do I get HTTP from a host using a specific IP address?
- For example, you may be trying out a website installation that is not yet in
- the DNS. Or you have a site using multiple IP addresses for a given host
- name and you want to address a specific one out of the set.
- Set a custom Host: header that identifies the server name you want to reach
- but use the target IP address in the URL:
- curl --header "Host: www.example.com" http://127.0.0.1/
- You can also opt to add faked host name entries to curl with the --resolve
- option. That has the added benefit that things like redirects will also work
- properly. The above operation would instead be done as:
- curl --resolve www.example.com:80:127.0.0.1 http://www.example.com/
- 3.20 How to SFTP from my user's home directory?
- Contrary to how FTP works, SFTP and SCP URLs specify the exact directory to
- work with. It means that if you do not specify that you want the user's home
- directory, you get the actual root directory.
- To specify a file in your user's home directory, you need to use the correct
- URL syntax which for SFTP might look similar to:
- curl -O -u user:password sftp://example.com/~/file.txt
- and for SCP it is just a different protocol prefix:
- curl -O -u user:password scp://example.com/~/file.txt
- 3.21 Protocol xxx not supported or disabled in libcurl
- When passing on a URL to curl to use, it may respond that the particular
- protocol is not supported or disabled. The particular way this error message
- is phrased is because curl does not make a distinction internally of whether
- a particular protocol is not supported (i.e. never got any code added that
- knows how to speak that protocol) or if it was explicitly disabled. curl can
- be built to only support a given set of protocols, and the rest would then
- be disabled or not supported.
- Note that this error will also occur if you pass a wrongly spelled protocol
- part as in "htpt://example.com" or as in the less evident case if you prefix
- the protocol part with a space as in " http://example.com/".
- 3.22 curl -X gives me HTTP problems
- In normal circumstances, -X should hardly ever be used.
- By default you use curl without explicitly saying which request method to
- use when the URL identifies an HTTP transfer. If you just pass in a URL like
- "curl http://example.com" it will use GET. If you use -d or -F curl will use
- POST, -I will cause a HEAD and -T will make it a PUT.
- If for whatever reason you are not happy with these default choices that curl
- does for you, you can override those request methods by specifying -X
- [WHATEVER]. This way you can for example send a DELETE by doing "curl -X
- DELETE [URL]".
- It is thus pointless to do "curl -XGET [URL]" as GET would be used
- anyway. In the same vein it is pointless to do "curl -X POST -d data
- [URL]"... But you can make a fun and somewhat rare request that sends a
- request-body in a GET request with something like "curl -X GET -d data
- [URL]"
- Note that -X does not actually change curl's behavior as it only modifies the
- actual string sent in the request, but that may of course trigger a
- different set of events.
- Accordingly, by using -XPOST on a command line that for example would follow
- a 303 redirect, you will effectively prevent curl from behaving
- correctly. Be aware.
- 4. Running Problems
- 4.2 Why do I get problems when I use & or % in the URL?
- In general Unix shells, the & symbol is treated specially and when used, it
- runs the specified command in the background. To safely send the & as a part
- of a URL, you should quote the entire URL by using single (') or double (")
- quotes around it. Similar problems can also occur on some shells with other
- characters, including ?*!$~(){}<>\|;`. When in doubt, quote the URL.
- An example that would invoke a remote CGI that uses &-symbols could be:
- curl 'http://www.altavista.com/cgi-bin/query?text=yes&q=curl'
- In Windows, the standard DOS shell treats the percent sign specially and you
- need to use TWO percent signs for each single one you want to use in the
- URL.
- If you want a literal percent sign to be part of the data you pass in a POST
- using -d/--data you must encode it as '%25' (which then also needs the
- percent sign doubled on Windows machines).
- 4.3 How can I use {, }, [ or ] to specify multiple URLs?
- Because those letters have a special meaning to the shell, to be used in
- a URL specified to curl you must quote them.
- An example that downloads two URLs (sequentially) would be:
- curl '{curl,www}.haxx.se'
- To be able to use those characters as actual parts of the URL (without using
- them for the curl URL "globbing" system), use the -g/--globoff option:
- curl -g 'www.site.com/weirdname[].html'
- 4.4 Why do I get downloaded data even though the web page does not exist?
- curl asks remote servers for the page you specify. If the page does not exist
- at the server, the HTTP protocol defines how the server should respond and
- that means that headers and a "page" will be returned. That is simply how
- HTTP works.
- By using the --fail option you can tell curl explicitly to not get any data
- if the HTTP return code does not say success.
- 4.5 Why do I get return code XXX from an HTTP server?
- RFC 2616 clearly explains the return codes. This is a short transcript. Go
- read the RFC for exact details:
- 4.5.1 "400 Bad Request"
- The request could not be understood by the server due to malformed
- syntax. The client SHOULD NOT repeat the request without modifications.
- 4.5.2 "401 Unauthorized"
- The request requires user authentication.
- 4.5.3 "403 Forbidden"
- The server understood the request, but is refusing to fulfill it.
- Authorization will not help and the request SHOULD NOT be repeated.
- 4.5.4 "404 Not Found"
- The server has not found anything matching the Request-URI. No indication
- is given as to whether the condition is temporary or permanent.
- 4.5.5 "405 Method Not Allowed"
- The method specified in the Request-Line is not allowed for the resource
- identified by the Request-URI. The response MUST include an Allow header
- containing a list of valid methods for the requested resource.
- 4.5.6 "301 Moved Permanently"
- If you get this return code and an HTML output similar to this:
- <H1>Moved Permanently</H1> The document has moved <A
- HREF="http://same_url_now_with_a_trailing_slash/">here</A>.
- it might be because you requested a directory URL but without the trailing
- slash. Try the same operation again _with_ the trailing URL, or use the
- -L/--location option to follow the redirection.
- 4.6 Can you tell me what error code 142 means?
- All curl error codes are described at the end of the man page, in the
- section called "EXIT CODES".
- Error codes that are larger than the highest documented error code means
- that curl has exited due to a crash. This is a serious error, and we
- appreciate a detailed bug report from you that describes how we could go
- ahead and repeat this.
- 4.7 How do I keep user names and passwords secret in curl command lines?
- This problem has two sides:
- The first part is to avoid having clear-text passwords in the command line
- so that they do not appear in 'ps' outputs and similar. That is easily
- avoided by using the "-K" option to tell curl to read parameters from a file
- or stdin to which you can pass the secret info. curl itself will also
- attempt to "hide" the given password by blanking out the option - this
- does not work on all platforms.
- To keep the passwords in your account secret from the rest of the world is
- not a task that curl addresses. You could of course encrypt them somehow to
- at least hide them from being read by human eyes, but that is not what
- anyone would call security.
- Also note that regular HTTP (using Basic authentication) and FTP passwords
- are sent as cleartext across the network. All it takes for anyone to fetch
- them is to listen on the network. Eavesdropping is easy. Use more secure
- authentication methods (like Digest, Negotiate or even NTLM) or consider the
- SSL-based alternatives HTTPS and FTPS.
- 4.8 I found a bug
- It is not a bug if the behavior is documented. Read the docs first.
- Especially check out the KNOWN_BUGS file, it may be a documented bug.
- If it is a problem with a binary you have downloaded or a package for your
- particular platform, try contacting the person who built the package/archive
- you have.
- If there is a bug, read the BUGS document first. Then report it as described
- in there.
- 4.9 curl cannot authenticate to a server that requires NTLM?
- NTLM support requires OpenSSL, GnuTLS, mbedTLS, Secure Transport, or
- Microsoft Windows libraries at build-time to provide this functionality.
- 4.10 My HTTP request using HEAD, PUT or DELETE does not work
- Many web servers allow or demand that the administrator configures the
- server properly for these requests to work on the web server.
- Some servers seem to support HEAD only on certain kinds of URLs.
- To fully grasp this, try the documentation for the particular server
- software you are trying to interact with. This is not anything curl can do
- anything about.
- 4.11 Why do my HTTP range requests return the full document?
- Because the range may not be supported by the server, or the server may
- choose to ignore it and return the full document anyway.
- 4.12 Why do I get "certificate verify failed" ?
- When you invoke curl and get an error 60 error back it means that curl
- could not verify that the server's certificate was good. curl verifies the
- certificate using the CA cert bundle and verifying for which names the
- certificate has been granted.
- To completely disable the certificate verification, use -k. This does
- however enable man-in-the-middle attacks and makes the transfer INSECURE.
- We strongly advise against doing this for more than experiments.
- If you get this failure with a CA cert bundle installed and used, the
- server's certificate might not be signed by one of the CA's in your CA
- store. It might for example be self-signed. You then correct this problem by
- obtaining a valid CA cert for the server. Or again, decrease the security by
- disabling this check.
- At times, you find that the verification works in your favorite browser but
- fails in curl. When this happens, the reason is usually that the server
- sends an incomplete cert chain. The server is mandated to send all
- "intermediate certificates" but does not. This typically works with browsers
- anyway since they A) cache such certs and B) supports AIA which downloads
- such missing certificates on demand. This is a server misconfiguration. A
- good way to figure out if this is the case it to use the SSL Labs server
- test and check the certificate chain: https://www.ssllabs.com/ssltest/
- Details are also in the SSLCERTS.md document, found online here:
- https://curl.se/docs/sslcerts.html
- 4.13 Why is curl -R on Windows one hour off?
- Since curl 7.53.0 this issue should be fixed as long as curl was built with
- any modern compiler that allows for a 64-bit curl_off_t type. For older
- compilers or prior curl versions it may set a time that appears one hour off.
- This happens due to a flaw in how Windows stores and uses file modification
- times and it is not easily worked around. For more details read this:
- https://www.codeproject.com/Articles/1144/Beating-the-Daylight-Savings-Time-bug-and-getting
- 4.14 Redirects work in browser but not with curl
- curl supports HTTP redirects well (see item 3.8). Browsers generally support
- at least two other ways to perform redirects that curl does not:
- Meta tags. You can write an HTML tag that will cause the browser to redirect
- to another given URL after a certain time.
- JavaScript. You can write a JavaScript program embedded in an HTML page that
- redirects the browser to another given URL.
- There is no way to make curl follow these redirects. You must either
- manually figure out what the page is set to do, or write a script that parses
- the results and fetches the new URL.
- 4.15 FTPS does not work
- curl supports FTPS (sometimes known as FTP-SSL) both implicit and explicit
- mode.
- When a URL is used that starts with FTPS://, curl assumes implicit SSL on
- the control connection and will therefore immediately connect and try to
- speak SSL. FTPS:// connections default to port 990.
- To use explicit FTPS, you use an FTP:// URL and the --ftp-ssl option (or one
- of its related flavors). This is the most common method, and the one
- mandated by RFC 4217. This kind of connection will then of course use the
- standard FTP port 21 by default.
- 4.16 My HTTP POST or PUT requests are slow
- libcurl makes all POST and PUT requests (except for requests with a small
- request body) use the "Expect: 100-continue" header. This header allows the
- server to deny the operation early so that libcurl can bail out before having
- to send any data. This is useful in authentication cases and others.
- However, many servers do not implement the Expect: stuff properly and if the
- server does not respond (positively) within 1 second libcurl will continue
- and send off the data anyway.
- You can disable libcurl's use of the Expect: header the same way you disable
- any header, using -H / CURLOPT_HTTPHEADER, or by forcing it to use HTTP 1.0.
- 4.17 Non-functional connect timeouts
- In most Windows setups having a timeout longer than 21 seconds make no
- difference, as it will only send 3 TCP SYN packets and no more. The second
- packet sent three seconds after the first and the third six seconds after
- the second. No more than three packets are sent, no matter how long the
- timeout is set.
- See option TcpMaxConnectRetransmissions on this page:
- https://support.microsoft.com/en-us/kb/175523/en-us
- Also, even on non-Windows systems there may run a firewall or anti-virus
- software or similar that accepts the connection but does not actually do
- anything else. This will make (lib)curl to consider the connection connected
- and thus the connect timeout will not trigger.
- 4.18 file:// URLs containing drive letters (Windows, NetWare)
- When using curl to try to download a local file, one might use a URL
- in this format:
- file://D:/blah.txt
- you will find that even if D:\blah.txt does exist, curl returns a 'file
- not found' error.
- According to RFC 1738 (https://www.ietf.org/rfc/rfc1738.txt),
- file:// URLs must contain a host component, but it is ignored by
- most implementations. In the above example, 'D:' is treated as the
- host component, and is taken away. Thus, curl tries to open '/blah.txt'.
- If your system is installed to drive C:, that will resolve to 'C:\blah.txt',
- and if that does not exist you will get the not found error.
- To fix this problem, use file:// URLs with *three* leading slashes:
- file:///D:/blah.txt
- Alternatively, if it makes more sense, specify 'localhost' as the host
- component:
- file://localhost/D:/blah.txt
- In either case, curl should now be looking for the correct file.
- 4.19 Why does not curl return an error when the network cable is unplugged?
- Unplugging a cable is not an error situation. The TCP/IP protocol stack
- was designed to be fault tolerant, so even though there may be a physical
- break somewhere the connection should not be affected, just possibly
- delayed. Eventually, the physical break will be fixed or the data will be
- re-routed around the physical problem through another path.
- In such cases, the TCP/IP stack is responsible for detecting when the
- network connection is irrevocably lost. Since with some protocols it is
- perfectly legal for the client to wait indefinitely for data, the stack may
- never report a problem, and even when it does, it can take up to 20 minutes
- for it to detect an issue. The curl option --keepalive-time enables
- keep-alive support in the TCP/IP stack which makes it periodically probe the
- connection to make sure it is still available to send data. That should
- reliably detect any TCP/IP network failure.
- TCP keep alive will not detect the network going down before the TCP/IP
- connection is established (e.g. during a DNS lookup) or using protocols that
- do not use TCP. To handle those situations, curl offers a number of timeouts
- on its own. --speed-limit/--speed-time will abort if the data transfer rate
- falls too low, and --connect-timeout and --max-time can be used to put an
- overall timeout on the connection phase or the entire transfer.
- A libcurl-using application running in a known physical environment (e.g.
- an embedded device with only a single network connection) may want to act
- immediately if its lone network connection goes down. That can be achieved
- by having the application monitor the network connection on its own using an
- OS-specific mechanism, then signaling libcurl to abort (see also item 5.13).
- 4.20 curl does not return error for HTTP non-200 responses
- Correct. Unless you use -f (--fail).
- When doing HTTP transfers, curl will perform exactly what you are asking it
- to do and if successful it will not return an error. You can use curl to
- test your web server's "file not found" page (that gets 404 back), you can
- use it to check your authentication protected web pages (that gets a 401
- back) and so on.
- The specific HTTP response code does not constitute a problem or error for
- curl. It simply sends and delivers HTTP as you asked and if that worked,
- everything is fine and dandy. The response code is generally providing more
- higher level error information that curl does not care about. The error was
- not in the HTTP transfer.
- If you want your command line to treat error codes in the 400 and up range
- as errors and thus return a non-zero value and possibly show an error
- message, curl has a dedicated option for that: -f (CURLOPT_FAILONERROR in
- libcurl speak).
- You can also use the -w option and the variable %{response_code} to extract
- the exact response code that was returned in the response.
- 5. libcurl Issues
- 5.1 Is libcurl thread-safe?
- Yes.
- We have written the libcurl code specifically adjusted for multi-threaded
- programs. libcurl will use thread-safe functions instead of non-safe ones if
- your system has such. Note that you must never share the same handle in
- multiple threads.
- There may be some exceptions to thread safety depending on how libcurl was
- built. Please review the guidelines for thread safety to learn more:
- https://curl.se/libcurl/c/threadsafe.html
- 5.2 How can I receive all data into a large memory chunk?
- [ See also the examples/getinmemory.c source ]
- You are in full control of the callback function that gets called every time
- there is data received from the remote server. You can make that callback do
- whatever you want. You do not have to write the received data to a file.
- One solution to this problem could be to have a pointer to a struct that you
- pass to the callback function. You set the pointer using the
- CURLOPT_WRITEDATA option. Then that pointer will be passed to the callback
- instead of a FILE * to a file:
- /* imaginary struct */
- struct MemoryStruct {
- char *memory;
- size_t size;
- };
- /* imaginary callback function */
- size_t
- WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data)
- {
- size_t realsize = size * nmemb;
- struct MemoryStruct *mem = (struct MemoryStruct *)data;
- mem->memory = (char *)realloc(mem->memory, mem->size + realsize + 1);
- if (mem->memory) {
- memcpy(&(mem->memory[mem->size]), ptr, realsize);
- mem->size += realsize;
- mem->memory[mem->size] = 0;
- }
- return realsize;
- }
- 5.3 How do I fetch multiple files with libcurl?
- libcurl has excellent support for transferring multiple files. You should
- just repeatedly set new URLs with curl_easy_setopt() and then transfer it
- with curl_easy_perform(). The handle you get from curl_easy_init() is not
- only reusable, but you are even encouraged to reuse it if you can, as that
- will enable libcurl to use persistent connections.
- 5.4 Does libcurl do Winsock initialization on win32 systems?
- Yes, if told to in the curl_global_init() call.
- 5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ?
- Yes, but you cannot open a FILE * and pass the pointer to a DLL and have
- that DLL use the FILE * (as the DLL and the client application cannot access
- each others' variable memory areas). If you set CURLOPT_WRITEDATA you must
- also use CURLOPT_WRITEFUNCTION as well to set a function that writes the
- file, even if that simply writes the data to the specified FILE *.
- Similarly, if you use CURLOPT_READDATA you must also specify
- CURLOPT_READFUNCTION.
- 5.6 What about Keep-Alive or persistent connections?
- curl and libcurl have excellent support for persistent connections when
- transferring several files from the same server. curl will attempt to reuse
- connections for all URLs specified on the same command line/config file, and
- libcurl will reuse connections for all transfers that are made using the
- same libcurl handle.
- When you use the easy interface the connection cache is kept within the easy
- handle. If you instead use the multi interface, the connection cache will be
- kept within the multi handle and will be shared among all the easy handles
- that are used within the same multi handle.
- 5.7 Link errors when building libcurl on Windows
- You need to make sure that your project, and all the libraries (both static
- and dynamic) that it links against, are compiled/linked against the same run
- time library.
- This is determined by the /MD, /ML, /MT (and their corresponding /M?d)
- options to the command line compiler. /MD (linking against MSVCRT dll) seems
- to be the most commonly used option.
- When building an application that uses the static libcurl library, you must
- add -DCURL_STATICLIB to your CFLAGS. Otherwise the linker will look for
- dynamic import symbols. If you are using Visual Studio, you need to instead
- add CURL_STATICLIB in the "Preprocessor Definitions" section.
- If you get a linker error like "unknown symbol __imp__curl_easy_init ..." you
- have linked against the wrong (static) library. If you want to use the
- libcurl.dll and import lib, you do not need any extra CFLAGS, but use one of
- the import libraries below. These are the libraries produced by the various
- lib/Makefile.* files:
- Target: static lib. import lib for libcurl*.dll.
- -----------------------------------------------------------
- MinGW: libcurl.a libcurldll.a
- MSVC (release): libcurl.lib libcurl_imp.lib
- MSVC (debug): libcurld.lib libcurld_imp.lib
- Borland: libcurl.lib libcurl_imp.lib
- 5.8 libcurl.so.X: open failed: No such file or directory
- This is an error message you might get when you try to run a program linked
- with a shared version of libcurl and your runtime linker (ld.so) could not
- find the shared library named libcurl.so.X. (Where X is the number of the
- current libcurl ABI, typically 3 or 4).
- You need to make sure that ld.so finds libcurl.so.X. You can do that
- multiple ways, and it differs somewhat between different operating systems.
- They are usually:
- * Add an option to the linker command line that specify the hard-coded path
- the runtime linker should check for the lib (usually -R)
- * Set an environment variable (LD_LIBRARY_PATH for example) where ld.so
- should check for libs
- * Adjust the system's config to check for libs in the directory where you have
- put the library (like Linux's /etc/ld.so.conf)
- 'man ld.so' and 'man ld' will tell you more details
- 5.9 How does libcurl resolve host names?
- libcurl supports a large number of name resolve functions. One of them is
- picked at build-time and will be used unconditionally. Thus, if you want to
- change name resolver function you must rebuild libcurl and tell it to use a
- different function.
- - The non-IPv6 resolver that can use one of four different host name resolve
- calls (depending on what your system supports):
- A - gethostbyname()
- B - gethostbyname_r() with 3 arguments
- C - gethostbyname_r() with 5 arguments
- D - gethostbyname_r() with 6 arguments
- - The IPv6-resolver that uses getaddrinfo()
- - The c-ares based name resolver that uses the c-ares library for resolves.
- Using this offers asynchronous name resolves.
- - The threaded resolver (default option on Windows). It uses:
- A - gethostbyname() on plain IPv4 hosts
- B - getaddrinfo() on IPv6 enabled hosts
- Also note that libcurl never resolves or reverse-lookups addresses given as
- pure numbers, such as 127.0.0.1 or ::1.
- 5.10 How do I prevent libcurl from writing the response to stdout?
- libcurl provides a default built-in write function that writes received data
- to stdout. Set the CURLOPT_WRITEFUNCTION to receive the data, or possibly
- set CURLOPT_WRITEDATA to a different FILE * handle.
- 5.11 How do I make libcurl not receive the whole HTTP response?
- You make the write callback (or progress callback) return an error and
- libcurl will then abort the transfer.
- 5.12 Can I make libcurl fake or hide my real IP address?
- No. libcurl operates on a higher level. Besides, faking IP address would
- imply sending IP packets with a made-up source address, and then you normally
- get a problem with receiving the packet sent back as they would then not be
- routed to you.
- If you use a proxy to access remote sites, the sites will not see your local
- IP address but instead the address of the proxy.
- Also note that on many networks NATs or other IP-munging techniques are used
- that makes you see and use a different IP address locally than what the
- remote server will see you coming from. You may also consider using
- https://www.torproject.org/ .
- 5.13 How do I stop an ongoing transfer?
- With the easy interface you make sure to return the correct error code from
- one of the callbacks, but none of them are instant. There is no function you
- can call from another thread or similar that will stop it immediately.
- Instead, you need to make sure that one of the callbacks you use returns an
- appropriate value that will stop the transfer. Suitable callbacks that you
- can do this with include the progress callback, the read callback and the
- write callback.
- If you are using the multi interface, you can also stop a transfer by
- removing the particular easy handle from the multi stack at any moment you
- think the transfer is done or when you wish to abort the transfer.
- 5.14 Using C++ non-static functions for callbacks?
- libcurl is a C library, it does not know anything about C++ member functions.
- You can overcome this "limitation" with relative ease using a static
- member function that is passed a pointer to the class:
- // f is the pointer to your object.
- static size_t YourClass::func(void *buffer, size_t sz, size_t n, void *f)
- {
- // Call non-static member function.
- static_cast<YourClass*>(f)->nonStaticFunction();
- }
- // This is how you pass pointer to the static function:
- curl_easy_setopt(hcurl, CURLOPT_WRITEFUNCTION, YourClass::func);
- curl_easy_setopt(hcurl, CURLOPT_WRITEDATA, this);
- 5.15 How do I get an FTP directory listing?
- If you end the FTP URL you request with a slash, libcurl will provide you
- with a directory listing of that given directory. You can also set
- CURLOPT_CUSTOMREQUEST to alter what exact listing command libcurl would use
- to list the files.
- The follow-up question tends to be how is a program supposed to parse the
- directory listing. How does it know what's a file and what's a directory and
- what's a symlink etc. If the FTP server supports the MLSD command then it
- will return data in a machine-readable format that can be parsed for type.
- The types are specified by RFC 3659 section 7.5.1. If MLSD is not supported
- then you have to work with what you are given. The LIST output format is
- entirely at the server's own liking and the NLST output does not reveal any
- types and in many cases does not even include all the directory entries.
- Also, both LIST and NLST tend to hide unix-style hidden files (those that
- start with a dot) by default so you need to do "LIST -a" or similar to see
- them.
- Example - List only directories.
- ftp.funet.fi supports MLSD and ftp.kernel.org does not:
- curl -s ftp.funet.fi/pub/ -X MLSD | \
- perl -lne 'print if s/(?:^|;)type=dir;[^ ]+ (.+)$/$1/'
- curl -s ftp.kernel.org/pub/linux/kernel/ | \
- perl -lne 'print if s/^d[-rwx]{9}(?: +[^ ]+){7} (.+)$/$1/'
- If you need to parse LIST output in libcurl one such existing
- list parser is available at https://cr.yp.to/ftpparse.html Versions of
- libcurl since 7.21.0 also provide the ability to specify a wildcard to
- download multiple files from one FTP directory.
- 5.16 I want a different time-out
- Sometimes users realize that CURLOPT_TIMEOUT and CURLOPT_CONNECTIMEOUT are
- not sufficiently advanced or flexible to cover all the various use cases and
- scenarios applications end up with.
- libcurl offers many more ways to time-out operations. A common alternative
- is to use the CURLOPT_LOW_SPEED_LIMIT and CURLOPT_LOW_SPEED_TIME options to
- specify the lowest possible speed to accept before to consider the transfer
- timed out.
- The most flexible way is by writing your own time-out logic and using
- CURLOPT_XFERINFOFUNCTION (perhaps in combination with other callbacks) and
- use that to figure out exactly when the right condition is met when the
- transfer should get stopped.
- 5.17 Can I write a server with libcurl?
- No. libcurl offers no functions or building blocks to build any kind of
- Internet protocol server. libcurl is only a client-side library. For server
- libraries, you need to continue your search elsewhere but there exist many
- good open source ones out there for most protocols you could want a server
- for. There are also really good stand-alone servers that have been tested
- and proven for many years. There is no need for you to reinvent them.
- 5.18 Does libcurl use threads?
- Put simply: no, libcurl will execute in the same thread you call it in. All
- callbacks will be called in the same thread as the one you call libcurl in.
- If you want to avoid your thread to be blocked by the libcurl call, you make
- sure you use the non-blocking multi API which will do transfers
- asynchronously - still in the same single thread.
- libcurl will potentially internally use threads for name resolving, if it
- was built to work like that, but in those cases it will create the child
- threads by itself and they will only be used and then killed internally by
- libcurl and never exposed to the outside.
- 6. License Issues
- curl and libcurl are released under a MIT/X derivative license. The license
- is liberal and should not impose a problem for your project. This section is
- just a brief summary for the cases we get the most questions. (Parts of this
- section was much enhanced by Bjorn Reese.)
- We are not lawyers and this is not legal advice. You should probably consult
- one if you want true and accurate legal insights without our prejudice. Note
- especially that this section concerns the libcurl license only; compiling in
- features of libcurl that depend on other libraries (e.g. OpenSSL) may affect
- the licensing obligations of your application.
- 6.1 I have a GPL program, can I use the libcurl library?
- Yes
- Since libcurl may be distributed under the MIT/X derivative license, it can
- be used together with GPL in any software.
- 6.2 I have a closed-source program, can I use the libcurl library?
- Yes
- libcurl does not put any restrictions on the program that uses the library.
- 6.3 I have a BSD licensed program, can I use the libcurl library?
- Yes
- libcurl does not put any restrictions on the program that uses the library.
- 6.4 I have a program that uses LGPL libraries, can I use libcurl?
- Yes
- The LGPL license does not clash with other licenses.
- 6.5 Can I modify curl/libcurl for my program and keep the changes secret?
- Yes
- The MIT/X derivative license practically allows you to do almost anything
- with the sources, on the condition that the copyright texts in the sources
- are left intact.
- 6.6 Can you please change the curl/libcurl license to XXXX?
- No.
- We have carefully picked this license after years of development and
- discussions and a large amount of people have contributed with source code
- knowing that this is the license we use. This license puts the restrictions
- we want on curl/libcurl and it does not spread to other programs or
- libraries that use it. It should be possible for everyone to use libcurl or
- curl in their projects, no matter what license they already have in use.
- 6.7 What are my obligations when using libcurl in my commercial apps?
- Next to none. All you need to adhere to is the MIT-style license (stated in
- the COPYING file) which basically says you have to include the copyright
- notice in "all copies" and that you may not use the copyright holder's name
- when promoting your software.
- You do not have to release any of your source code.
- You do not have to reveal or make public any changes to the libcurl source
- code.
- You do not have to broadcast to the world that you are using libcurl within
- your app.
- All we ask is that you disclose "the copyright notice and this permission
- notice" somewhere. Most probably like in the documentation or in the section
- where other third party dependencies already are mentioned and acknowledged.
- As can be seen here: https://curl.se/docs/companies.html and elsewhere,
- more and more companies are discovering the power of libcurl and take
- advantage of it even in commercial environments.
- 7. PHP/CURL Issues
- 7.1 What is PHP/CURL?
- The module for PHP that makes it possible for PHP programs to access curl-
- functions from within PHP.
- In the cURL project we call this module PHP/CURL to differentiate it from
- curl the command line tool and libcurl the library. The PHP team however
- does not refer to it like this (for unknown reasons). They call it plain
- CURL (often using all caps) or sometimes ext/curl, but both cause much
- confusion to users which in turn gives us a higher question load.
- 7.2 Who wrote PHP/CURL?
- PHP/CURL was initially written by Sterling Hughes.
- 7.3 Can I perform multiple requests using the same handle?
- Yes - at least in PHP version 4.3.8 and later (this has been known to not
- work in earlier versions, but the exact version when it started to work is
- unknown to me).
- After a transfer, you just set new options in the handle and make another
- transfer. This will make libcurl re-use the same connection if it can.
- 7.4 Does PHP/CURL have dependencies?
- PHP/CURL is a module that comes with the regular PHP package. It depends on
- and uses libcurl, so you need to have libcurl installed properly before
- PHP/CURL can be used.
- 8. Development
- 8.1 Why does curl use C89?
- As with everything in curl, there is a history and we keep using what we have
- used before until someone brings up the subject and argues for and works on
- changing it.
- We started out using C89 in the 1990s because that was the only way to write
- a truly portable C program and have it run as widely as possible. C89 was for
- a long time even necessary to make things work on otherwise considered modern
- platforms such as Windows. Today, we do not really know how many users that
- still require the use of a C89 compiler.
- We will continue to use C89 for as long as nobody brings up a strong enough
- reason for us to change our minds. The core developers of the project do not
- feel restricted by this and we are not convinced that going C99 will offer us
- enough of a benefit to warrant the risk of cutting off a share of users.
- 8.2 Will curl be rewritten?
- In one go: no. Little by little over time? Maybe.
- Over the years, new languages and clever operating environments come and go.
- Every now and then the urge apparently arises to request that we rewrite curl
- in another language.
- Some the most important properties in curl are maintaining the API and ABI
- for libcurl and keeping the behavior for the command line tool. As long as we
- can do that, everything else is up for discussion. To maintain the ABI, we
- probably have to maintain a certain amount of code in C, and to remain rock
- stable, we will never risk anything by rewriting a lot of things in one go.
- That said, we can certainly offer more and more optional backends written in
- other languages, as long as those backends can be plugged in at build-time.
- Backends can be written in any language, but should probably provide APIs
- usable from C to ease integration and transition.
|