Evaluation Methodology

Why formalise tech stack changes?

• Comprehensive stakeholder engagement

• Future-Proofing solutions

• Minimise misalignment and duplication

• Accelerated development processes

• Improved risk management

• Rigorous due diligence

• Better resource allocation

Does / API Data Extraction need an evaluation?

• Cross-cutting need for extracting data from APIs (as opposed to niche use case)

• APIs are evolving (REST vs GraphQL)

• Multiple stakeholders involved (allow analysts to extract data from APIs?)

• Needs to integrate with existing data and analytics engineering tech stack

• Alternatives are available e.g. Meltano

• dltHub can be used for more than extracting data from APIs so there is a risk of expansion and duplication

Evaluation methodology

Evaluation methodology

Modified from 2023 Iceberg Evaluation and team guide to RFCs

Pre-RFC

Personas / Users

Evaluation Criteria

Data Engineer Story Mapping

Next Steps

There are still lots of open questions:

1. How to ensure proportionality?
2. How to enforce standards without stifling innovation?
3. What is the role of the Centre for Excellence?
4. When to flip from investigation to evaluation?
5. Where to record this information?
6. How do other teams e.g. Data Science deal with x-domain tech changes?
7. How to integrate with other evaluation forums e.g. solution surgery?