How to Run a Execute cURL Commands in Python

I love quick wins in the terminal. Type a single line, hit Enter, and data flows back. But sooner or later I need that same call inside a Python project maybe to chain it with other logic, maybe to share it with the team. A few mornings ago I had to do exactly that for Google’s QPX Express API error failed with a “Parse Error.” Here’s the full story, the fix, and a handful of extras you can bolt on to make the script production-ready.

The Command Show:

curl -d @request.json --header "Content-Type: application/json" https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere

There is a request.json file to be sent to get a response.

I searched a lot and got confused. I tried to write a piece of code, although I could not fully understand it and it didn’t work.

import pycurl
import StringIO

response = StringIO.StringIO()
c = pycurl.Curl()
c.setopt(c.URL, 'https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere')
c.setopt(c.WRITEFUNCTION, response.write)
c.setopt(c.HTTPHEADER, ['Content-Type: application/json','Accept-Charset: UTF-8'])
c.setopt(c.POSTFIELDS, '@request.json')
c.perform()
c.close()
print response.getvalue()
response.close()

The error message is Parse Error. How to get a response from the server correctly?

The Bash One Liner

-d @request.json \
     -H "Content-Type: application/json" \
     "https://www.googleapis.com/qpxExpress/v1/trips/search?key=<API_KEY>"

• -d @request.json tells cURL to read request.json off disk and POST its bytes.
• -H "Content-Type: application/json" sets the header.
• The URL carries my API key as a query string.

Flawless in Bash. Now for Python.

Running Code

pycurl
from io import StringIO      # whoops—Python 3 mismatch

response = StringIO()
curl = pycurl.Curl()
curl.setopt(curl.URL,
            'https://www.googleapis.com/qpxExpress/v1/trips/search?key=<API_KEY>')
curl.setopt(curl.HTTPHEADER,
            ['Content-Type: application/json', 'Accept-Charset: UTF-8'])
curl.setopt(curl.POSTFIELDS, '@request.json')   # ← the trap
curl.setopt(curl.WRITEFUNCTION, response.write)
curl.perform()
curl.close()

print(response.getvalue())

Why the Parse Error?

pycurl does not treat @request.json as “read this file.” It passes the literal string to the server. Google’s endpoint sees @request.json, tries to parse it as JSON, panics, and sends back a 400 with “Parse error.”

Two extra gotchas:

1. I imported StringIO from the wrong place for Python 3 (io.StringIO instead).
2. I never checked the HTTP status code, so the failure looked like silence until that cryptic error landed.

Fix Pycurl Version

json
import os
import pycurl
from io import BytesIO

def post_qpx_request(json_file: str, api_key: str) -> dict:
    if not os.path.isfile(json_file):
        raise FileNotFoundError(f"{json_file} is missing")

    with open(json_file, "rb") as fp:
        payload = fp.read()

    buf = BytesIO()
    curl = pycurl.Curl()
    curl.setopt(pycurl.URL,
                f"https://www.googleapis.com/qpxExpress/v1/trips/search?key={api_key}")
    curl.setopt(pycurl.HTTPHEADER,
                ['Content-Type: application/json', 'Accept-Charset: UTF-8'])
    curl.setopt(pycurl.POST, 1)
    curl.setopt(pycurl.POSTFIELDS, payload)
    curl.setopt(pycurl.WRITEFUNCTION, buf.write)

    try:
        curl.perform()
        status = curl.getinfo(pycurl.RESPONSE_CODE)
    finally:
        curl.close()

    if status != 200:
        raise RuntimeError(f"HTTP {status}: {buf.getvalue()[:200]}")

    return json.loads(buf.getvalue())

if __name__ == "__main__":
    result = post_qpx_request("request.json", "<API_KEY>")
    print(json.dumps(result, indent=2))

Change Explain

Result: clean JSON back from the API.

The Requests Library

Unless you truly need libcurl’s speed or multi-handle tricks, requests is easier to read and maintain.

json
import requests

with open("request.json") as fp:
    body = json.load(fp)

resp = requests.post(
    "https://www.googleapis.com/qpxExpress/v1/trips/search",
    params={"key": "<API_KEY>"},
    json=body,          # auto-serialises & sets header
    timeout=15
)

resp.raise_for_status()  # throws on bad codes
print(resp.json())

I timed both: requests was maybe 10 ms slower on my machine irrelevant for a network round-trip. So that’s what I ship.

Extras to Level Up

Pick one tonight, add the snippet, commit, and your quick script turns into a tiny but solid client.

Final Thought

Most “Python vs Bash” headaches boil down to tiny differences in how special characters are handled. Here the villain was a single @ that the shell understood but pycurl did not. My cure was simple: read the file, hand the raw bytes to the library or skip pycurl and use requests so I never worry about it again.