Coming from FME¶
duck soup covers the most common kind of FME workspace: read a few datasets, join and geoprocess them, reshape the attributes to a target schema, and write the result. This page maps the FME ideas and transformers you already know to their duck soup equivalents.
How the model differs¶
| FME | duck soup |
|---|---|
A workspace (.fmw) |
A config (a YAML file), built in the editor or by hand |
| Readers | Sources |
| Writers | The config's output file and each pipeline's output layers |
| A canvas where features flow along many connections | A pipeline: one base source flowing through an ordered list of steps |
| Several unrelated flows in one workspace | Several pipelines in one config, writing into the same output |
| Routing features to different writers with a Tester | One layer filter per output layer |
| A side branch that feeds back in later | A snapshot branch |
| Inspector / feature caching | The editor's live preview, on every step |
| Feature counts on the connections | count rows in the pipeline flow, on a sample or all the data |
| Passed / Failed (Unmatched, Outside…) output ports | rejects on spatial, attribute and nearest-neighbour joins, clip and filter: the failed rows go to a layer of their own |
| The canvas, and a transformer's parameters opened from it | The pipeline flow ("ducks in a row"): click a node and its card opens in a panel beside it |
| Coordinate system on each reader and writer | One working CRS for the processing, plus a CRS per source and per layer |
Every row in a duck soup pipeline starts as one feature of the base source. Other sources are
reference data that steps join against, so a "merge these two readers" flow becomes a
merge step, and a "match readers A and B" flow becomes a join
with A as the base.
Transformers¶
Joins and overlays¶
| FME transformer | duck soup |
|---|---|
| PointOnAreaOverlayer, SpatialRelator | spatial_join (predicate: intersects / contains / within). Their match counts and lists: match: summarize |
| SpatialFilter | spatial_join with rejects: matches go on as Passed, the rest to the rejects layer as Failed |
| FeatureMerger, DatabaseJoiner, Joiner | attribute_join; a FeatureMerger's list of all matches: match: summarize |
| NeighborFinder | nearest_neighbor |
| AreaOnAreaOverlayer, LineOnAreaOverlayer | intersect_overlay |
| LineOnLineOverlayer | line_overlay, one step per overlay dataset |
| HTTPCaller (one request per feature) | http_lookup: {column} in the URL in place of @Value(), and the JSON response's values picked as fields, so no JSONFlattener afterwards |
| HTTPCaller (one file for all features) | A source with an http(s) uri, joined with attribute_join |
A spatial join keeps base features that match nothing, like a FeatureMerger's Unmerged
output joined back in. To route them to their own layer instead, like the Unmerged port,
set rejects: on the join. To drop them, follow the join with a filter step.
Geometry¶
| FME transformer | duck soup |
|---|---|
| Bufferer | buffer step, or a derived source with buffer: |
| CenterPointReplacer | centroid |
| Clipper (inside output) | clip; its rejects layer is the features entirely outside |
| Clipper (outside output) | erase |
| Dissolver, Aggregator | dissolve, with aggregates for the attribute accumulation (sum, count, concatenate, …) |
| LineCombiner | combine_lines: group by attributes, one line per connected chain, aggregates per chain |
| Generalizer (Douglas, Visvalingam) | simplify; shared_boundaries: true for polygons that must keep fitting together |
| CoordinateRounder | snap_to_grid |
| Snapper (polygons onto each other), AreaGapAndOverlapCleaner | clean_coverage |
| Deaggregator | explode, with part_field for the part number |
| DonutHoleExtractor (filled outer shapes) | fill_holes, with max_area for only the small holes |
| Intersector, TopologyBuilder (noding lines) | split_lines; with source, lines cut at another layer's lines or polygon edges |
| SliverRemover, AreaGapAndOverlapCleaner (small areas) | eliminate_slivers: polygons under an area merged into a neighbour |
| GeometryCoercer, BoundaryExtractor, BoundingBoxReplacer, HullReplacer, InsidePointReplacer | An attributes step setting geom with an expression: ST_Boundary(geom), ST_Envelope(geom), ST_ConvexHull(geom), ST_PointOnSurface(geom), … |
| GeometryValidator (repair) | make_valid, on by default for every source |
| 2DForcer | force_2d: true on the source |
| Reprojector | Automatic: sources are reprojected into the working CRS, and layers into their own crs |
Rows¶
| FME transformer | duck soup |
|---|---|
| Tester, TestFilter | filter step (drops rows, or with rejects writes them to a Failed layer), or a layer filter (splits rows across layers) |
| Several readers into one stream | merge |
| DuplicateFilter, Matcher (identical geometry) | remove_duplicates: by columns, geometry: true, and rejects for the Duplicate port |
Attributes¶
Attribute work happens in the mapping, which lists every output
column in order. Anything not mapped isn't written, which replaces most AttributeKeeper and
AttributeRemover use. When a step needs a column that doesn't exist yet (a key to dissolve or
join on, a value to filter by), an attributes step
(Update Attributes in the editor) sets it
partway through the pipeline. It takes the same rules as the mapping, plus drop, like an
AttributeManager placed between transformers.
| FME transformer | duck soup mapping |
|---|---|
| AttributeRenamer, AttributeKeeper | {to: new_name, from: old_name} |
| AttributeCreator (fixed value) | {to: source, const: "Survey 2026"} |
| AttributeManager, ExpressionEvaluator, StringConcatenator, StringCaseChanger, StringReplacer | expr with a DuckDB SQL expression |
| AttributeValueMapper, ValueMapper | codelist, with rules or a CSV lookup |
| AttributeTypeConverter | cast |
| UUIDGenerator | func: uuid |
| TimeStamper, DateTimeStamper | func: now / func: today |
| CoordinateExtractor (of a centre point) | func: lon / func: lat, or func: mgrs |
| AreaCalculator, LengthCalculator | func: area / func: length |
| GeometryExtractor (WKB) | func: wkb |
Different attribute sets for different writers become a per-layer mapping.
Running and scheduling¶
| FME | duck soup |
|---|---|
| Running from FME Workbench | run in the editor, or duck-soup run config.yaml |
fme.exe workspace.fmw in a batch file |
duck-soup run config.yaml. See Command line. |
| FME Flow (Server) schedules | cron, systemd, Task Scheduler, Docker or CI. See Scheduling runs. |
| FME Flow job history | The editor's History tab, which lists scheduled runs too |
| Viewing what a transformer does under the hood | The editor's SQL tab |
What duck soup doesn't do¶
- Output formats: only GeoPackage and GeoParquet. There are no writers for Shapefile, databases or web services.
- Published parameters: the YAML has no variables or parameters. Keep one config per variant, or generate the YAML with a script.
- PythonCaller and custom transformers: there's no scripting hook. Most attribute logic
fits in a DuckDB SQL
expr. - Feature-by-feature control flow: loops, looping custom transformers and feature-level routing beyond filters aren't available. A pipeline is a fixed chain of steps.
- Raster and topology tools: duck soup works with vector features and tables.
See also Known limitations.