A list has been created on the Beam website: https://beam.apache.org/contribute/design-documents/
- Overall model
- Streams and Tables https://s.apache.org/beam-streams-tables
- Streaming SQL http://s.apache.org/streaming-sql-spec
- Removing the Pipeline object: https://s.apache.org/no-beam-pipeline
- Dynamic DoFn https://s.apache.org/a-new-dofn
- Splittable DoFn (Obsoletes Source API) http://s.apache.org/splittable-do-fn
- Reimplementing Beam API classes on top of Splittable DoFn on top of Source API https://s.apache.org/sdf-via-source
- Watch transform http://s.apache.org/beam-watch-transform
- Technical contract for runtime rebalancing of work using Splittable DoFn https://s.apache.org/beam-breaking-fusion
- State and Timers for DoFn: https://s.apache.org/beam-state
- Side inputs and Side outputs in MapElements: http://s.apache.org/context-fn
- Triggers https://s.apache.org/beam-triggers
- Triggering is for sink https://s.apache.org/beam-sink-triggers (Not implemented)
- Lateness https://s.apache.org/beam-lateness
- Metrics
- Metrics API http://s.apache.org/beam-metrics-api
- I/O Metrics https://s.apache.org/standard-io-metrics
- Progress https://s.apache.org/beam-fn-api-progress-reporting
- Graphical view of progress https://docs.google.com/document/d/1Dx18qBTvFWNqwLeecemOpKfleKzFyeV3Qwh71SHATvY/edit#
- Checkpoints https://s.apache.org/FIWQ
- Runner API https://s.apache.org/beam-runner-guide (obsoletes
http://s.apache.org/beam-runner-api) - Runners use the Fn API to tell the underlying execution engine how to call back into user code http://s.apache.org/beam-fn-api
- Letting pipelines in language X run on runners in language Y https://docs.google.com/document/d/1U7daqDBIYsQdmJcKck1yCv0Httva8UVoH5BsFUofZuo/edit#heading=h.bp6lkv9dhu2o https://s.apache.org/beam-mixed-language-pipelines
- Developer experience
There may be other technical design documents in this google drive folder: https://goo.gl/ps8twC including:
