High Performance ETL Service

A configurable event-driven data processing system

In this customer project, an event-driven program for retrieving and analyzing external data was developed. The data is obtained from various public sources. The customer can use configuration entries to set the source system to be queried. Depending on the configuration, data is retrieved cyclically or event-driven via API, web sockets and webhooks. The information obtained is filtered using rules. These are configurable and can trigger further actions. The processed data is imported into a database and further processed by an existing customer system.

Holistic solution development

DTQ Solutions not only developed the desired software in time, budget and scope, but also supported its customers holistically in creating the solution. This includes, for example, selecting a hosting company that meets the customer’s specific functional requirements – of course, taking into account essential criteria such as price and reliability. Setting up the software artifact on the target system or configuring and connecting additional systems are also part of the solution. It is also a matter of course for us to provide practical documentation and introductory training as part of the system handover.

“DTQ Solutions GmbH has already proven itself to be a reliable and competent partner in several projects. Our requirements have always been implemented to a high standard, extremely quickly and to our complete satisfaction.

The service is highly professional and flexible at a fair price; We recommend DTQ Solutions without reservation.

We would particularly like to emphasize the extremely good accessibility, short response times and the always binding and friendly communication at eye level.”

Wernberg & Partner GbR, Mai 2023

Technology

In this project we used the following technologies: Agile software development with Python, MySQL, API, web socket and webhook connection, event driven programming, test-driven development (TDD), basic data analytics procedures.