Generative AI And Semantic Compliance

Republished By Plato

Followers: 0

Only CPT and its peers know how many statements have been made based on results from generative AI. But there are loads of them.

My background as a data modeler over many years makes me shiver a little bit, because what the friendly AI helpers help us produce is subjected to cognitive processes, where we, the readers, process the input, the AI results, with regards to apprehended semantics (by ourselves), specific knowledge that each of us has built over time. And we expect the AI results to follow similar (i.e., corresponding) patterns.

In other words, I would expect “ChatSome” to implicitly be compliant with specific semantics, built into the training sets:

*Diagram made by way of WebWOWL – go there to read the diagram*

The semantic network in the above example is the well-known FOAF (Friend of a Friend) ontology, which comes out of the world wide web community. So, if ChatSome discusses friend-related issues, it would seem natural to expect semantic structures not too different from the above standard (FOAF).

Shared semantics is a necessity in human communication.

Prompt Engineering to the Rescue?

Seems that the answer, for now, from the AI marketeers, is careful, curated, “prompt engineering” – because if you cannot define what you are asking about, how should any AI (or human) be able to answer intelligently?

See, for example, this excellent recent article from Harvard Business Review by Oguz A. Acar, “AI Prompt Engineering Isn’t the Future”:

I encourage you to read the article, but you will get the gist of it in this citation:

“However, despite the buzz surrounding it, the prominence of prompt engineering may be fleeting for several reasons. First, future generations of AI systems will get more intuitive and adept at understanding natural language, reducing the need for meticulously engineered prompts. Second, new AI language models like GPT4 already show great promise in crafting prompts – AI itself is on the verge of rendering prompt engineering obsolete. Lastly, the efficacy of prompts is contingent upon the specific algorithm, limiting their utility across diverse AI models and versions.

So, what is a more enduring and adaptable skill that will keep enabling us to harness the potential of generative AI? It is problem formulation – the ability to identify, analyze, and delineate problems.”

I agree with that a long way. It means that there is more work to do. In fact, it seems to me that what we hear from some of the most optimistic and hopeful generative AI market makers is on par with this famous (alleged) quote by the Swedish author Astrid Lindgren, who, supposedly, lets her prime character, Pippi Longstocking say: “I have never tried that before, so I think I should definitely be able to do that.”

Here is a book cover from the Pippi Longstocking universe:

Nothing was too difficult for Pippi, even lifting her horse! There are at least 14 books and a number of movies documenting this. Children and their parents are fascinated by these stories well into adulthood. Why does this remind me of some of the narratives coming out of parts of California?

However, more balanced views are available: In his excellent new book, “The DataPreneurs – The Promise of AI and the Creators Building Our Future,” Bob Muglia highlights how Yann LeCun of New York University and Meta points out some fundamentals about learning and apprehension in “A Path Towards Autonomous Machine Intelligence Version 0.9.2, 2022-06-27”:

“Human and non-human animals seem able to learn enormous amounts of background knowledge about how the world works through observation and through an incomprehensibly small amount of interactions in a task-independent, unsupervised way. It can be hypothesized that this accumulated knowledge may constitute the basis for what is often called common sense. Common sense can be seen as a collection of models of the world that can tell an agent what is likely, what is plausible, and what is impossible. Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation.”

Common sense is certainly worth looking at, as the following little chat shows.

A Recent Chat

Recently I chatted with GPT 4 about data modeling. I asked about the “25 MOST Important Data Model Types.”

When the result came back, I immediately noticed that Fact Modeling was not on the list. As soon as I remarked that, the result was modified, and now it looked like this:

“I apologize for the omission. The Fact Model is indeed an important data modeling technique, used to model complex systems and capture business rules. It focuses on identifying and defining the facts that describe the operations of a system, as well as the relationships between those facts.

Therefore, the Fact Model should also be included in the list of important data models. Here is an updated list of the 25 most important data models, including the Fact Model:

Relational Data Model

Entity-Relationship (ER) Data Model

Object-Oriented Data Model

Object-Relational Data Model

Hierarchical Data Model

Network Data Model

Dimensional Data Model

Flat-File Data Model

Semantic Data Model

Conceptual Data Model

Logical Data Model

Physical Data Model

Data Warehouse Data Model

Data Mart Data Model

XML Data Model

NoSQL Data Model

Document-Oriented Data Model

Column-Family Data Model

Key-Value Data Model

Graph Data Model

RDF Data Model

OWL Data Model

UML Data Model

EAV Data Model

OLAP Data Model

Fact Model

Thank you for bringing it to my attention.”

Me: Are data vault and anchor models also important data model types?

“Yes, both data vault and anchor models are important types of data models that have gained popularity in recent years.

Free Research Preview. ChatGPT may produce inaccurate information about people, places, or facts.”

***End of chat with ChatGPT Mar 23 Version.

Yes, it did produce inaccurate information. And it added content to the result in an additional “prompt engineering” step.

I am tempted to say: Pippi, do your homework!

Life Can Only Be Understood Backwards, But It Must Be Lived Forwards

The subheading above is a quote from the Danish philosopher Soren Kierkegaard:

*Image source: The Royal Library, Denmark, 1923, public domain, via Wikimedia Commons*

Large language models (LLMs) are, by necessity, products of the past facts about past events. But we, as generative AI consumers, need forwardly reliable predictions. Even given that all the “transformers” are old news.

Understanding both past and future contexts is of essence. In fact, generative AI and its results are an exercise in context sensitivity – in other words, semantics, meaning, structure, relationships, over time.

That is the real schisma.

Running generative predictions based on slightly outdated LLM content is like running on a data warehouse from last year.

Adding predictions on top of LLMs is also known as statistics. That is all well and fine, in particular, if you are provided with some quality and confidence indicators.

Needs for Normative Semantic Standards?

We, as professionals (data modelers and information scientists), must take some responsible action to force the generative AI tools to be able to constrain themselves to a given context and semantics.

It could be an organization’s own data models and glossary, but it could also be industry or country or EU official ontologies and vocabularies, as well as schema.org and Wikidata and so forth.
We need the AI results to exhibit clearly defined contexts and show compliance with defined semantics.

There are many possibilities in this space.

First and foremost, I would like to know:

Is this result based on entirely documentable, established facts? Or can fictional semantics and assertions occur? In other words, is this fact or fiction? That is close to insisting on a general label meaning “Contains certified facts only”!

If I were in a heavily regulated sector such as government, finance, or pharma, I would like to know the confidence with which I can trust in the results. Will they hold in court, for example? Are they multi-temporal (or at least bitemporal)?

Looking at quality management in general (see, for example, the ISO 9000 QMS set of standards) there are many aspects of ISO 9000 that Data Management could learn from. So, a generative AI system could be labeled according to the quality perspective that it complies with. This could lead to an ISO semantic compliance management standard? Similar to the ISO 9000 family.

Another important aspect is specificity. Think the FOAF ontology that we started out with. In fact, there are so many international and national standards, vocabularies, and industry data models that it would make sense to introduce a compliance statement on AI results.

Here, follow a few examples of “normative” (or at least commonly used) semantic media, useful for mapping meaning on the Internet taken from Andrew Iliadis’s recent book with that title (andrewiliadis.com):

The above are just a few examples.

Also on the national and international levels there are plenty of publicly available ontologies, taxonomies, data models, schemas, and what have you.

Well, actually, what you also have, are some home brew data models of your own. Wouldn’t it be nice to have a compliance testing between your own semantics and those of a generative AI LLM? Could be a nice AI product feature, if you ask me.

Verifiably Compliant Generative AI

If I was working in the financial sector, I might well want to know whether the proposed result, as well as the language model, was certified to comply with the vocabulary and semantics of FIBO (the financial industry business ontology published by the EDM council). If I can count on that, I am home safe. If not, I have to read, interpret, and draw additional conclusions – together with bunches of colleagues in the regulatory compliance department. In the financial sector that staff is already being directed down this road, with the BCBS 239 standard asking for integrated taxonomies and architectures.

Not only the results, but also the transformers and other parts of language models, must be verifiably compliant.

I honestly do believe that the world needs quality assurance at these levels, if business and politics are to be conducted based on generative AI at a larger scale. So, bring in those certified AI generators! And give us, the people with HI, a chance!