Gernot Bernkopf
on June 5th, 2023
dcupl Pre-Processing
Uncover the benefits of using dcupl pre-processing to get the right data for your applications. Cut your data horizontally and vertically based on target models and by means of dcupl's complex, nested queries. Input data is read in and rewritten in a correspondingly reduced form.
TL;DR - Data is an essential component in the digital world, powering applications and tools that help businesses provide better services to their customers. However, not all data is relevant or necessary for every application - and no API is a universal tool which can fulfill all use cases out-of-the-box. New applications and services have new data requirements and thus usually require new APIs or at least new endpoints. A challenge in terms of time and budget. The good news is that pre-processing based on dcupl’s search and filter technology can help you to quickly obtain the right data for your application.
Why pre-processing?
dcupl’s sophisticated search and filter technology allows for easy server- and client-side data filtering. Based on this, dcupl offers data pre-processing, with the help of which data can be cut horizontally and vertically. Based on target models and by means of complex, nested queries, input data is read in and rewritten in a correspondingly reduced form.
Every database does that? That's right, of course. But with dcupl pre-processing, it's not about sending a query and delivering the result via API, but the "pre" is decisive. The dcupl pre-processing in turn pays into our conceptual advantages in the development process of applications. With pre-processing, we provide another tool that supports the Front-end First approach and our principle of two speeds - Dual-Speed.
Advantages of pre-processed data
- Decrease amount of data to the necessary minimum
- Reduce server-side calculation time
- Reduce network traffic
- Use the right data and data amounts
- Start developing instantly with real data
- Save money for interfaces that are not absolutely necessary
An example from a current project
A customer approached one of our partners with the requirement to digitize a business process that is based on data from the ERP and has to interact with it; a rollout in Q3/2023 was formulated as the central requirement, although no interface to the ERP existed for the planned project so far.
Perfect scenario for the use of dcupl
Since the ERP system does not yet provide an adequate interface and the development takes a longer period of time, the data required for the application was exported from the ERP - statically as CSV. The data sets with the company's articles and services contain up to 35,000 records and up to 1,900 properties. Quantities that naturally far exceed the required number of records and the scope of properties in the target application.
This is where dcupl pre-processing comes in
With the help of target models and rules based on the dcupl search and filter engine, the exports provided are cut horizontally and vertically and new target files are created for the future application. These contain only those data records and properties that are needed by the user in the future application. Defining the target models and filter rules and generating the new, optimized datasets took one day.
dcupl ensures the most convenient and cost-efficient way of providing data for the customer. The export of the data is easy and quick to manage; no attributes have to be selected or extensive settings made for the export in the ERP - or any other system. Once set up in dcupl, pre-processing runs as often as needed; manually triggered, at intervals or as part of the continuous integration workflows.
For those data resources whose frequency of change is low, the export and pre-processing will be retained in the future. In a parallel process, interfaces are developed for entities that change frequently and therefore need to be dynamically loaded in production - dual-speed, without delaying the development of the application.
The process of using dcupl’s search and filter technology for pre-processing is straightforward. After exporting all data and columns from your ERP or PIM system, you would define filter rules based on dcupl. By doing this, only relevant data is retained while eliminating unnecessary data. Subsequently, the resulting dataset is perfect for application needs, and the application would provide impressive performance results. Pre-processing can be done locally or remotely, depending on the requirements of the application. If the application is required to run the pre-processing at intervals, it is best to perform pre-processing server-side respectively using the dcupl Console.
We would be happy to discuss this point with you. Just book a 30-minute meeting and we will try to outline a solution together with you.
Conclusion
Achieving digital excellence requires access to relevant data, and pre-processing based on dcupl’s search and filter technology is a great way to get there. With the ability to filter data based on specific values and cut data horizontally and vertically, businesses can generate customized datasets that are perfect for their application. This technology can help businesses manage their data more effectively by creating data sets that contain only relevant information, reducing processing times and improving overall efficiency.
By using dcupl, you can ensure that your web applications are secure, scalable, and fast-loading, giving your users the best possible experience. So, whether you're a CEO, CMO, CIO, technical director, dev lead, or security expert, dcupl is a tool you should explore if you want to streamline your development process and create fast loading web application.
Ready to get started?
We look forward to working with you and your team, no matter if you are a SME or a global enterprise. dcupl helps enterprises to develop efficiently by minimizing risks effortlessly.