<!--order=6-->

<h1>Python: Calling Web APIs with urllib</h1>
<p>Many services provide data through a web API. Python can make simple HTTP requests using the standard library, which is useful on a small Debian machine where keeping dependencies to a minimum matters.</p>

<h2>Make a GET Request</h2>
<pre>
import json
from urllib.request import Request, urlopen

url = "https://api.github.com/repos/python/cpython"
request = Request(
    url,
    headers={"User-Agent": "python-learning-example"},
)

with urlopen(request, timeout=10) as response:
    data = json.load(response)

print(data["full_name"])
print(data["stargazers_count"])
</pre>
<p>The server may reject requests without a user agent, and every network request should have a timeout so a disconnected field node does not wait forever.</p>

<h2>Handle Network Errors</h2>
<pre>
from urllib.error import HTTPError, URLError

try:
    with urlopen(request, timeout=10) as response:
        data = json.load(response)
except HTTPError as error:
    print(f"Server returned HTTP {error.code}")
except URLError as error:
    print(f"Could not reach the server: {error.reason}")
</pre>
<p>Do not hide every exception with a blanket <code>except Exception</code>. Handling the errors you expect makes failures easier to diagnose.</p>

<h2>Query Parameters</h2>
<pre>
from urllib.parse import urlencode

params = urlencode({"q": "raspberry pi", "page": 1})
url = "https://example.invalid/search?" + params
print(url)
</pre>
<p>Use <code>urlencode</code> rather than placing user input directly into a URL. It correctly escapes spaces and other special characters.</p>

<h2>Respect the Service</h2>
<ul>
  <li>Read the API documentation before writing a client.</li>
  <li>Use timeouts and handle temporary failures.</li>
  <li>Do not poll more often than the service allows.</li>
  <li>Cache data when it does not need to be fresh every second.</li>
  <li>Never put passwords or API keys directly in source code.</li>
</ul>

<h2>Decode Only What You Need</h2>
<p>API responses can contain much more data than the program needs. Check that the fields you depend on are present and keep the conversion close to the request:</p>
<pre>
def repository_summary(data):
    return {
        "name": data.get("full_name", "unknown"),
        "stars": int(data.get("stargazers_count", 0)),
        "language": data.get("language") or "not listed",
    }
</pre>
<p>Using <code>get()</code> is appropriate for optional fields. For a required field, raise a clear error instead of quietly producing an incomplete report.</p>

<h2>Retry Temporary Failures Carefully</h2>
<pre>
import time
from urllib.error import URLError

for attempt in range(3):
    try:
        with urlopen(request, timeout=10) as response:
            data = json.load(response)
        break
    except URLError:
        if attempt == 2:
            raise
        time.sleep(2 ** attempt)
</pre>
<p>Retries are for temporary network failures, not for invalid credentials or a malformed request. Keep the number of attempts small so an offline Pi does not spend all day retrying.</p>

<h2>Keep Secrets Out of Source</h2>
<p>If an API needs a token, read it from an environment variable or a protected configuration file:</p>
<pre>
import os

token = os.environ.get("SERVICE_TOKEN")
if not token:
    raise RuntimeError("SERVICE_TOKEN is not configured")
</pre>
<p>Do not put real tokens in examples, Git repositories, shell history, or messages sent over the mesh.</p>

<h2>References</h2>
<ul>
  <li><a href="https://docs.python.org/3/library/urllib.request.html" target="_blank" rel="noopener">Python urllib.request ↗</a></li>
  <li><a href="https://docs.python.org/3/library/urllib.parse.html" target="_blank" rel="noopener">Python urllib.parse ↗</a></li>
</ul>