Intelligent experiences in your stack: GenAI architecture considerations

There is a transformative wave of generative AI digital products being designed, engineered and integrated into business operations and customer service in Q4 2023. This upsurge in intelligent digital products will reshape how businesses serve, operate and personalise their interactions across health, energy & financial services.

As user adoption surges and we witness a proliferation of new services, many businesses will need to look at their information architecture, ways of working and approach to CI/CD.

Whilst many people will be looking to the front end and thinking about the transformational experiences, we know that there needs to be a profound shift in how organisations address the pertinent concerns of a new product paradigm, where rather than digital products iterating at release windows, they are potentially iterating in real-time, with real-world consequences… from risk mitigation to data protection, experience fidelity to communications continuity, the world is changing and software development needs to change too.

Let’s take a look at five considerations that you need to interface with.

Whilst working in collaboration with our clients and engaging with our own development community, I have come to establish five key areas that businesses need to address when they are in the process of setting their AI strategy and creating the hypotheses for deployment across their organisation. Let’s explore each area in turn:

Ensuring your enterprise readiness

A truly holistic adoption of Gen AI into your digital product stack requires that your chosen foundation models align with your enterprise frameworks, and meet the exacting standards of fidelity, security, reliability, and corporate responsibility.

I believe that organisations must focus their initial efforts on robustly identifying and focusing their AI governance models, ensuring that the integration of Gen AI doesn’t in any way compromise their enterprise security, Integration and interoperability frameworks especially when businesses are looking to enable full-stack solutions.

Enterprise readiness doesn’t stop at governance, to be truly enterprise-ready, businesses need to have established trust in the AI. In the process of establishing trust, they must look at how their work culture and ways of working may be informed by adopting computer intelligence into the operating modality of their wider organisation.

Finally, the readiness of the organisation requires a new perspective on product development. These technologies are new (at least in this context) and therefore require a wholly new approach to how product teams iterate and experiment to realise their goals. Make no doubts about it, leveraging Gen AI is nothing like leveraging previously emerging technologies (think mobile, voice etc.) and teams will need to adapt to the highly iterative nature of how this new development process works.

Accessibility of environments & models

For many organisations, the key component of establishing trust will be establishing full control over the experimental and development environments in which hypotheses are tested and business value proven.

Whether you choose to deploy large language models on your own public cloud infrastructure, on private infrastructure or as a managed cloud service from an external vendor, you will need to consider a wide range of factors including identifying and managing the right infrastructure, navigating the landscape of different model families, version controlling models, developing associated talent and skills, developing full-stack services for easier adoption and much, much more.

The landscape of models that are available is vast, and so is the challenge of assessing the strengths and weaknesses of each model for its particular use case. A production-ready LLM-powered application is in reality an orchestrated collection of multiple pipelines and components, some containing traditional logic and others containing specialist niche models that perform specific tasks.

Regardless of the decisions, these choices will inevitably guide the potential success of intelligent product development, so the accessibility and availability of the right environments are going to be key to fully realising business value.

Our advice is to start with a stack that is aligned to your existing internal skills for experimentation (i.e. if you’re an Azure house, start there!) Once your teams have a feel for which experiments succeed and which don’t, they can move onto deeper topics like model evaluation and fine tuning.

MLops and managing pipelines

Gen AI is now moving into the production limelight after spending years in the experimental/research phase. To streamline deployments businesses and the development community have already started maturing processes to create standardised ML pipelines that are optimised for scale, efficiency, and control.

A problem in todays teams is the large gap between data scientists and software product teams in terms of how they think and operate.

But when looking at Gen AI in particular it means that the vast majority of models used will be ‘pre-trained’. That’s what the ‘P’ in GPT means. So getting started with Gen AI is not a data project in itself and secondly, enterprises have evolved towards a more controlled environment for ML pipelines by implementing MLOps — a lifecycle management technique for machine learning and artificial intelligence solutions.

MLOps facilitates communication between development and operations teams by implementing a set of standardised practices across the entire ML pipeline, starting from requirements gathering, all the way to managing observability in production. It is the foundation to ensure the scale, speed, and quality of ML solutions.

As Gen AI intelligent digital products start to be deployed, the MLOps function will inevitably morph and change, into a discipline more oriented to the intricacies of Generative AI from more traditional deep, machine learning.

Customisation with Proprietary Data and Knowledge

For many businesses, the initial key opportunity for leveraging the power of large language models comes in the form of creating internal, business-specific toolkits that will aid their own people in becoming more efficient and efficacious.

While there are a lot of perspectives on harnessing company data we believe the capabilities of LLMs unlock a new paradigm of harnessing company knowledge and intelligence. Whether that be about your products and services or how you best support customer needs, businesses need to think of this rich natural language content as the new era of data, that is human readable and relevant to colleagues and customers.

The two are not mutually exclusive however. For example, a company could use business intelligence to identify trends in customer feedback and then use generative AI to create personalised responses. This combination of data-driven insights and AI-generated content can improve customer experience and operational efficiency.

Ethical & Environmental Considerations

I have chosen to pool these areas for the purpose of this article, but be under no illusions both require your full attention and have massive implications for your business, your people and your customers.

On the question of environmental considerations, whilst being pre-trained, foundation models can have substantial energy (and in some cases cost) demands, especially during customisation phases. This means that your carbon footprint can grow significantly and this can have knock-on implications for your CSR, sustainability or Net Zero initiatives.

As businesses scale their Gen AI applications, the potential environmental impact becomes a crucial consideration. A strategic approach is required to ensure sustainable adoption without adversely affecting the organisation’s carbon footprint.

When we switch to think about AI ethics, we bump up against much bigger questions of how utilising Gen AI maps to your business values and principles, alongside understanding the concerns of the people you are serving, the people whose data has been used to train the LLMs and the intricacies of an AI-enabled world & workplace.

This is too large a conversation to cover in one paragraph, but needless to say, it is one that keeps me up at night.

Conclusion

The integration of Gen AI into your business presents a transformative opportunity for organisations across of core verticals of energy, health and financial services. Whilst the emphasis on selecting the right model and technology is paramount, the true key to success lies in the holistic approach: focusing on both the architecture, people and processes involved.

As the technology landscape evolves rapidly, agility and adaptiveness will be the cornerstones for businesses looking to fully harness the potential of this AI paradigm shift. If any of the themes in this article have provoked questions in your mind about your own business, why not reach out to our team to carry on the discussion?