Almost 30 years since the advent of the web, some are still committed to returning to original web principles that many others have forgotten. At the same time, the committed ones are working on solving the data and architecture problems enterprises have allowed to fester and grow to the scale of an epidemic since the first days of ERP in the 1970s.
Helping to solve intractable problems
What problems? Data architect Robert Hanson alluded to one of these seemingly intractable enterprise data issues in an August 2022 post on LinkedIn:
“Our data warehouse platforms are bleeding us dry, not because of computing costs. At least not directly….
“And it’s not compute costs that are the problem, it’s the people costs….
“Then we get back to the data engineer. I don’t know if you noticed, but none are available. Try hiring one today. That’s because they’re all employed building secondary pipelines populating summary tables that BI analysts asked for. After all, the platform is too slow. Every summary pipeline takes about two person weeks. That’s expensive.
“All because our platforms don’t aggregate or join efficiently.
“We need to demand more from our data platforms, it’s costing us a fortune.”
Webby data architects and modelers–the spider-like ones who use intelligent graph design and a bit of glue or another sticky substance to achieve their objectives–are focused on making joinery much more efficient and scaling a lot more useful with the help of more contextualized data. They consider what they’re doing essential to real progress. It’s obvious to them.
By making data self-describing and designed to connect, they’re designed to make sure even the heterogeneous bits fit together (the structured and the less so, for example), so they can interact the way they’re supposed to when they’re first put together.
The webby way (now N-dimensional semantic graph) lends itself to the efficiencies of many-to-many construction.
Careful design implies up-front work, but investing the time early means less agony and expense down the road. Struggling with aggregation after the fact is more expensive, as the 1:10:100 rule of thumb of Total Quality Management reminds us. Prevention might cost $1, while correction or remediation might cost $10. Failure might well cost $100.
Quite a few of the less perishable kinds of business, scientific and medical data require this kind of care and nurturing. For example, the pharmaceutical and biomedical industries have long created shared taxonomies and ontologies out of necessity. Now they’re the ones who can share more broadly and discoverably, with more confidence and trust than other industries.
If you’re building for advanced analytics and machine learning, the many-to-many and adaptive means of semantic graphs are uniquely sustainable. Without this sort of investment, your staff will likely be spending most of their time reconfirming that the data you thought you had isn’t as good as it needs to be for replicability and thus for sound decision-making.
A Polymath Crowd at the Data-Centric Architecture Forum
Photo by Maksim Romashkin
“For too long, we’ve believed in a single path to excellence,” said author Dan Pink.
David Epstein in Range, the book Pink was praising when Pink commented, talks about conceptual versus procedural math. The conceptual variety is harder to grasp, at least initially. For this reason, teachers at US schools tend to gravitate …….
Source: https://www.datasciencecentral.com/semantic-graph-as-the-next-step-for-web-data-architecture/