# XML to CSV feed publisher

This runs on the feed server. The website remains responsible for resolving dealers, updating/inserting adverts, and deactivating missing stock. No database connection is required on this server.

## Run

PHP 8.3 with curl and SimpleXML is available on this server. No Composer dependencies.

```sh
# Fetch and validate only (default); does not change the published feed.
php /var/www/feeds/xml_feed_importer/run.php autocraft
# Fetch, validate, generate CSV and publish its JSON file list.
php /var/www/feeds/xml_feed_importer/run.php autocraft --publish
# Process all enabled config entries.
php /var/www/feeds/xml_feed_importer/run.php --publish
# Regression tests (isolated temporary output).
php /var/www/feeds/xml_feed_importer/tests/run.php
```

Configuration is in `config.php`. Dealer keys are the website's exact `dealer_feed_reference`, not a numeric dealer ID. The supplied key is `autocraftmotorcaravans`, per the request. confirm the database reference before website integration. Other feeds can use additional keys if they implement the same XML schema. Different XML schemas require a mapping adapter.

## Published files

Example paths (timestamps and random suffix vary):

```
xml_feed_importer/json/autocraft_csv_file_list.json
xml_feed_importer/uploads/xml_csv/autocraft/2026/10/autocraft_20261004_230000_ab12cd34ef56.csv
xml_feed_importer/uploads/xml_csv/autocraft/2026/10/autocraft_20261004_230000_ab12cd34ef56.csv.manifest.json
```

The JSON index is an Agenty-style array containing the latest CSV basename. Derive the year/month from that filename, not the current date. The sidecar manifest has `dealer_feed_reference`, `complete`, `row_count`, `csv_file`, `relative_path`, `sha256`, `generated_at`, and `schema_version`.

Each CSV is generated in a temporary file in its target directory, flushed, and renamed. The immutable CSV and sidecar are completed before the index is atomically replaced. That index rename is the publication commit. Consumers must use the index, not glob for the newest file. A failed batch leaves the previous index and its CSV intact. Unreferenced files from an interrupted publication are harmless; no automated retention cleanup is installed.

A per-dealer lock spans fetching, parsing and publication. Downloads require HTTPS, TLS validation and HTTP 200, with configured timeout, limited redirects, and a 20 MiB limit. Parsing validates the complete document, schema version, declared count, required identity/price/status fields, and unique IDs and URLs. Empty, truncated, count-mismatched, duplicate, invalid-price, and unknown-status batches are rejected entirely. DTD/entity declarations are rejected.

A provider that returns a syntactically complete but mistakenly reduced count cannot be identified from XML alone. The website must apply a stock-drop guard before deactivation, as described in WEBSITE_HANDOFF.md.

## Mapping

The headers retain the populated historical Autocraft Agenty CSV columns, with `advert_url` and `dealer_feed_reference` appended. Import by header name.

| XML | CSV |
| --- | --- |
| url | url AND advert_url (individual product URL) |
| id | vehicle_id (preserved as a string) |
| title | vehicle_name |
| price | vehicle_price (numeric GBP, no currency symbol) |
| year | vehicle_year_of_registration |
| mileage | vehicle_mileage (source text) |
| colour | vehicle_colour |
| engine_size | vehicle_engine (litres; 2200 becomes 2.2) |
| bhp | vehicle_bhp (source text) |
| gearbox | vehicle_transmission |
| fuel | vehicle_fuel_type |
| berth | vehicle_berth |
| base_van_features | vehicle_specification |
| images/image | vehicle_image_gallery (comma and space separated) |
| first images/image | featured_image |
| conversion_features | conversion |
| description | detail |
| config key | dealer_feed_reference |
| Autocraft dealer category | vehicle_type = Motorhome |

`status` must be `Available`; other values reject the batch until their mapping is explicitly agreed. Currency must be GBP. Missing optional source fields remain empty. Descriptions retain newlines and quotes using standard CSV escaping. All source images, including promotional graphics, are retained. XML length, extras/handover text, and video have no corresponding historical CSV columns and are not imported. Make/model are not invented from the title.

## Operations

`logs/import.log` contains JSON lines with dealer reference, time, status, row count, batch metadata or error. Exit status is nonzero if any enabled dealer fails. No old Agenty config, webhook, database, or stock-removal cron is changed by this publisher.

Once the website integration and switch-over are complete, install a scheduled job as a user with write access to this directory, for example:

```cron
15 2 * * * /usr/bin/php /var/www/feeds/xml_feed_importer/run.php --publish >> /var/www/feeds/xml_feed_importer/logs/cron.log 2>&1
```

Schedule is an example, not installed. Use log rotation appropriate to this host.

## FastCron URL endpoint

`endpoint.php` accepts GET or POST and publishes all enabled feeds. Authentication is required through `?token=...` or the `X-Feed-Token` header. The generated secret lives in `endpoint-secret.php` (not printed in application logs). Keep the full URL private; query tokens can appear in HTTP access logs. Optional `&dealer=autocraftmotorcaravans` selects one enabled dealer.

Print the complete URL on the server, then paste it into FastCron:

```sh
php /var/www/feeds/xml_feed_importer/show-fastcron-url.php
```

Schedule daily at 23:00 in the intended FastCron timezone, ahead of the website import. Use a request timeout longer than the configured feed timeout (at least 90 seconds for the single 60-second feed). Adding multiple dealers increases the total required request duration; use separate dealer-specific jobs if necessary. Web-server timeout settings also apply.

HTTP 200 with status `ok` means publication succeeded; 403 means invalid authentication, 400 means unknown/disabled dealer, and 500 means the run failed (including lock contention). Existing per-dealer locks prevent simultaneous CLI/HTTP publication. `run.php` remains CLI-only. The endpoint does not perform website database imports. No FastCron job is created automatically.

Output directories use the www-data group and group inheritance so CLI and web runs can share output and lock files. After copying/restoring files, retain these permissions. The live endpoint was tested for missing/wrong tokens, unknown dealers, and a successful publication; CLI validation and the 39 converter checks also pass.