A Whiskey Metaphor for AI Challenges

At a recent gathering of Hong Kong's tech leaders during the event titled "Mastering AI Blueprint: Building AI-Driven Smart Digital Infrastructure," Chris Barford, EY’s AI and data lead for financial services in Hong Kong, drew a striking comparison between AI models and whiskey distillation: selecting the right Large Language Models (LLMs) can be as nuanced as crafting the perfect blend of whiskey. However, this smooth analogy belies a pressing reality—Hong Kong's financial services sector is facing an urgent and complicated technological overhaul.

The Data Governance Dilemma

Barford wasted no time addressing Hong Kong’s unique challenges, notably the intricate expectations around data usage in the Greater China region. "Consumer patterns indicate a growing demand for ethical data practices outside of Mainland China," he said, spotlighting the delicate balance between staying compliant with regulations and pushing for innovation. He contextualized this struggle amid evolving geopolitical tensions, which complicate access to vital hardware for running AI applications.

LLM Agnostic Architecture: The Future of AI

The solution, as outlined by Barford, is an innovative framework known as an 'LLM agnostic architecture.' This structure allows financial institutions to deploy multiple models while adapting to the ever-changing landscape of costs, quality, and regulations. "As AI models evolve rapidly," Barford noted, "the ability to pivot is essential for success in this space. We are on the cusp of achieving a highly refined understanding of customer profiles through generative AI capabilities, which were previously unattainable."

The Energy Challenge: Are We Ready?

Steven So, NTT's newly appointed CTO for Asia, presented a sobering reality check concerning infrastructure. He highlighted startling figures: AI factories today consume between 40-50 megawatts, with individual GPU racks pulling up to 80 kilowatts—significantly higher than traditional enterprise settings. "In the next five years, AI will account for 3% of global energy use," So projected, underscoring an urgent need for sustainable power solutions.

Cooling Down: A Revolutionary Shift in AI Data Centers

But it’s not just about power; cooling technology is also undergoing a revolution. So showcased NTT's Direct Liquid Cooling (DLC) systems, which drastically improve heat management in data centers, making them essential as GPU densities rise. He emphasized how this technology is already operational in various regions, including India and Hong Kong, preparing for a future where AI chipsets demand unprecedented cooling solutions.

Towards a Photonic Future

So's bold vision includes NTT’s IOWN (Innovative Optical and Wireless Network) initiative, aiming to transform digital infrastructure by merging photonic and electronic technologies. This could enhance connectivity while reducing power consumption and latency—critical factors for industries like autonomous driving and high-frequency trading. "We anticipate ultra-fast transmission capacities and reduced latencies that could redefine the financial services landscape in Hong Kong," he asserted.

Humans and AI: A New Customer Experience

The panel discussion revealed how organizations are implementing these technologies in practical ways. Owen Chong from Xiaoice shared how some Chinese banks now deploy AI-driven digital humans for account opening processes, creating a futuristic scenario where human interaction is largely replaced by AI.

Custom Solutions for a Unique Market

Phil McManus from NTT further elaborated that there is no universal solution for the financial sector's AI requirements. Increased demand for local hosting and AI models has led NTT to adapt its infrastructure to be modular and scalable, meeting the unique needs of its clients. He underscored the growing inclination for private AI models to address regulatory concerns.

AI: The Double-Edged Sword

Ryan Manuel, founder of Bilby AI, pointed out the inherent challenges in deploying AI in highly regulated financial sectors. He urged industry leaders to focus on innovative applications of AI that could attract more customers, as opposed to merely replacing existing roles.

A Sustainability Focus in Infrastructure

NTT's commitment to sustainable infrastructure is notable, with plans to achieve net zero emissions by 2040. This long-term perspective is essential for an industry facing skyrocketing energy demands. Manuel noted that organizations need to reassess what they excel in and adapt accordingly, especially as AI continues to evolve.

Seizing the Hong Kong Advantage

Hong Kong's unique position as a bridge between East and West provides a significant advantage for its financial services. However, success will hinge on partnerships with technology providers who are committed to groundbreaking research and development. With NTT investing $3 billion annually in R&D, the future of AI infrastructure looks promising.

The Future is Fluid

The urgency for adaptability is paramount. As Barford observed, the pace of change in AI is too fast for companies to stick with outdated procurement cycles. The firms that will thrive are those that innovate, embrace emerging technologies like photonic computing, and construct flexible, sustainable infrastructures. Like master distillers, Hong Kong’s tech leaders must blend various elements into a sophisticated concoction that leads to richer outcomes—will you be among the first to take a sip?