Ernie.SG 
My point is: “杀猪焉用牛刀”, or, you don’t need a sledgehammer to crack open a nut. The unbearable lightness of large language models are that they are not optimised to a particular problem. While multi-modal models like GPT-4 and what was used in the search engine I prototyped are exciting, I’m watching with abated breath to see how engineers, scientists and researchers would code out an analogous human brain to the extreme.
4 months ago, I made use of supervised and unsupervised learning methods with 2 team mates to do automated annotation of artworks. At the end of it, I got interested in building MLOps pipelines to move from Proof-of-Concept to production and there were zero results on the entire internet if you were to look up “MLOps for museums” back then.
Now, Google will point you here and I am writing this post just to strengthen this keyword density and connection, hehe.
The interesting thing to me about making one month’s worth of (part-time) team effort obsolete with less than a half day’s work when I ditched supervised learning and unsupervised learning methods for building on top of OpenAI’s Contrastive Language-Image Pre-Training and Jina’s neural search capabilities instead is the difference in people’s reception towards search vis-à-vis automated annotation. Somehow, we are a lot more forgiving in the former and will take results as they come. In the latter, people seem perturbed by the idea of machine-made representations that puzzles me. Maybe we ascribe greater accountability and trust in human-generated knowledge (however fallible human memory might be, however limited in perspective) because there’s another being like us embedded within particular cultures and contexts that can be held responsible, while we conveniently discount the fact that such humans probably have varying degrees of technology use to begin with so the real puzzle is: what is really stopping us from taking representations of the world as informed by a particular model, constructed a particular way by a specific human agent? Maybe we just don’t like what we see in this mirror held up against ourselves, reflecting back at us our deepest biases and prejudice. We rather look the other way. We rather continue constructing castles on sand rather than venture and find new land.
Interesting, most of us seem to prefer the return of deterministic results over stochastic optimisations set over iterations. I wonder if this seemingly isolated observations ties back to parts of our human psychology nicely, or perhaps I am overfitting.
Anyway, search is interesting because it is a well-established, huge market opportunity and I happened to be running up against limits with the efficacy of Google when I was doing some searches. Also the Mandarin internet and search in the Mandarin language, how do I even begin…? While segments of the internet population live in a 21st century Cultural Revolution, the real limits of search in its current form and GPT-x are susceptibility to censorship and the deluge of trash. Attribution and better truth-checking mechanisms will not automatically make more people more interested in the truth, or to find common ground across multi-verses; still I think it’s important to make such mechnisations available as polder against the rising tide of falsehoods and hallucinations. In this world upon us where the cost of creation is close to zero (or $0.002 per 1000 ChatGPT tokens, cheaper by 10x if you access via API so what other reasons do you have for NOT learning coding!? :p), I’m excited to see how these technological forces will reshape creativity and usher in new paradigms, new business models and new modalities of expression and organisation that was hitherto impossible. Will Keynesian possibilities for our grandchildren find renewed relevance in such a world? Or will this Infinity Gauntlet doom half of the world to nihilism at best, and revolt at worst as work (and its associated meanings, however untenable the link) evaporates into thin air?
Like most things in life, there is probably a trade-off between the generalisability of the current models that we are seeing and specificity, and it will take a lot of experimentation to figure the right balance out when all the dust is done and settled.
Source: The life cycle of a technological revolution from “Technological Revolutions and Financial Capital” by Carlota Perez.
So for now, I’ll continue building. 😊
Originally published on PubPub at erniesg.pubpub.org/pub/ettc9fqh.