Python: Files, Paths, and JSON

Reading and writing files is where a small Python program starts becoming useful. The pathlib module handles paths clearly, while JSON is a convenient format for configuration and small data files.

Working with Paths

from pathlib import Path

reports = Path("reports")
reports.mkdir(exist_ok=True)

report_file = reports / "latest.txt"
report_file.write_text("Node is online\n", encoding="utf-8")

print(report_file.exists())
print(report_file.read_text(encoding="utf-8"))

Using Path instead of manually joining strings keeps the program portable across operating systems.

Finding Files

from pathlib import Path

for path in Path(".").glob("**/*.log"):
    print(path)

The ** pattern searches through subdirectories as well. Be careful with scripts that modify every matching file until you have tested the pattern.

Writing JSON

import json
from pathlib import Path

settings = {
    "node_name": "field-node-01",
    "frequency": 915,
    "enabled": True,
}

Path("settings.json").write_text(
    json.dumps(settings, indent=2) + "\n",
    encoding="utf-8",
)

Reading JSON

import json
from pathlib import Path

settings = json.loads(
    Path("settings.json").read_text(encoding="utf-8")
)

print(settings["node_name"])
print(settings.get("location", "not set"))

dict.get() is useful when an optional setting may not exist. For an important setting, access it directly so a missing value fails clearly.

Safer File Programs

  • Use with open(...) or Path.read_text() so files are closed properly.
  • Specify an encoding, normally utf-8, when reading text.
  • Write to a temporary file before replacing an important configuration file.
  • Never build a shell command from untrusted file names.

Handling Missing or Invalid Files

import json
from pathlib import Path

def load_settings(filename):
    try:
        text = Path(filename).read_text(encoding="utf-8")
        return json.loads(text)
    except FileNotFoundError:
        print(f"{filename}: file does not exist")
    except json.JSONDecodeError as error:
        print(f"{filename}: invalid JSON at line {error.lineno}")
    return {}

settings = load_settings("settings.json")

Configuration is external input, even when it is stored on the same machine. Report a useful error and choose a safe default instead of silently continuing with half a configuration.

CSV for Simple Tables

import csv
from pathlib import Path

with Path("readings.csv").open(newline="", encoding="utf-8") as file:
    for row in csv.DictReader(file):
        temperature = float(row["temperature"])
        print(row["node"], temperature)

Use JSON when the data is nested or configuration-shaped. CSV is convenient for flat rows that need to be opened in a spreadsheet.

Safe Replacement of a Configuration File

If a program is rewriting an important file, write the complete new content first and replace the old file only after the write succeeds:

from pathlib import Path

target = Path("settings.json")
temporary = target.with_suffix(".json.tmp")
temporary.write_text(
    '{\n  "enabled": true\n}\n',
    encoding="utf-8",
)
temporary.replace(target)

This reduces the chance of leaving an empty or half-written file after a power interruption. For more demanding applications, also keep a timestamped backup.

References