Samsung has appointed Lin Jun-cheng, who worked at its foundry business rival TSMC, who will serve as a senior vice president of the advanced packaging team under Samsung’s chip business division, device solutions. 

Read more: What Does TSMC’s $40 Billion Investment Mean for Chipmakers?

Jun-cheng, who was in TSMC for 19 years, played a significant role in the development of 3D packaging technology at the Taiwan-based chip-making giant. Before joining Samsung, he served as the chief of Skytech, a semiconductor equipment firm in Taiwan. Prior to his tenure at TSMC, Lin worked for US-based Micron Technology which specialises in memory semiconductors.

Lin’s recruitment by Samsung coincides with the company’s strong investment in advanced packaging technology, an area where it has lagged behind global competitors like TSMC and Intel.

Samsung’s High Stakes Bet on Talent Falters as Chip Market Takes a Hit

Kim Woo-pyeong, who previously worked at Apple, was appointed as the head of Samsung’s Packaging Solution Center at device solution last year. In addition, Samsung recruited Benny Katibian, a self-driving chip expert who previously worked for Qualcomm, to improve its self-driving technology. Samsung Research also recently hired Kwon Jung-hyun, a former Nvidia engineer, for robotics research. 

However, Samsung and SK Hynix, are facing difficult situations due to declining memory prices and tighter US restrictions on China. In February, South Korea’s exports fell 75%  to $50.1 billion, with semiconductor exports dropping by 42.5%. South Korea’s exports to China also fell by 24.2% due to weak demand for chips and petrochemical products. 

Samsung is expected to incur an operating loss of up to KRW 4 trillion from its memory chip business in the first quarter of 2023, the first operating loss since the global financial crisis in 2008. The device solutions division, which includes memory, IC, and foundry businesses, accounts for more than half of Samsung’s operating income. In addition, the US-China tension is a significant geopolitical risk for chipmakers, and South Korea’s chip inventory levels have increased by 28% in January compared to the previous month.

Read more: India: A Dumping Ground for Global Semiconductor Waste?

The post Samsung Scores Coup by Hiring Former TSMC Exec appeared first on AIM.

The global data center semiconductor and component market skyrocketed an unprecedented 152 percent in the first quarter of 2024, marking a new milestone, according to a report from Dell’Oro Group.

This explosive growth was fueled by insatiable demand for GPUs and custom accelerators, particularly in the hyperscale cloud sector.

The report revealed that in Q1 2024, NVIDIA led all vendors in component revenues, accounting for nearly half of the reported figures, as supplies of its H100 GPUs improved for both cloud and enterprise markets. Samsung and Intel followed NVIDIA in the rankings.

Looking ahead, strong growth for accelerators is expected to continue into 2024, with GPUs remaining the primary choice for AI training and inference workloads. NVIDIA’s upcoming Blackwell platform is poised to strengthen the firm’s leadership position.

However, the report anticipates that custom accelerators and offerings from other vendors, such as the AMD MI300X/MI325X Instinct and Intel Gaudi3, will gain some market share.

The report also noted that revenues for Smart NICs and DPUs surged more than 50 percent in Q1 2024, driven by strong hyperscale adoption for both AI and non-AI use cases. Storage drives and memory saw significant price increases as vendors aimed to align supply with demand. The three major memory suppliers shifted production capacity from DRAM to AI-focused High Bandwidth Memory (HBM) products.

Baron Fung, Senior Research Director at Dell’Oro Group, highlighted, “Accelerators such as GPUs continue to drive substantial growth, with shipments hitting record highs each quarter. Meanwhile, traditional server and storage component markets returned to positive year-over-year growth as vendors and cloud service providers ramped up purchases in anticipation of robust system demand later this year.”

General-purpose computing components also rebounded strongly following an inventory correction cycle in 2023, experiencing double-digit revenue growth. “Average selling price (ASP) of components has increased significantly from a year ago adding to topline growth,” Fung explained.

“For CPUs, an increasing mix toward fourth- and fifth-generation CPUs, which have more cores and feature sets compared to their predecessors, have commanded higher ASPs.”

As data centers continue to expand and evolve to support the explosive growth of AI and cloud computing, the demand for high-performance semiconductors and components shows no signs of slowing down.

The post NVIDIA Blackwell Solidify Leadership, AMD & Intel to Gain Ground With MI300X & Gaudi3 appeared first on AIM.

At Computex 2024, Intel has officially announced details about its forthcoming Lunar Lake chips, set to power Copilot+ AI PCs this fall. These new chips will deliver up to 48 TOPS (tera operations per second) of AI performance, supported by an upgraded neural processing unit (NPU). 

This represents a significant leap from Intel’s previous Meteor Lake chips, which offered a 10 TOPS NPU, and positions Intel in the ongoing AI performance race against competitors like AMD and Qualcomm.

Unveiled at Computex, Intel’s Lunar Lake chips promise substantial advancements. Alongside the impressive AI performance, they will feature a new Xe2 GPU, providing 80 percent faster gaming performance compared to the previous generation. 

Additionally, an AI accelerator in the chip will contribute an extra 67 TOPS of performance. Despite these enhancements, Intel faces competition from AMD’s Ryzen AI 300 chips, launching in July with 50 TOPS NPUs, and Qualcomm’s Snapdragon X Elite and X Plus chips. These competitors highlight the aggressive push within the AI PC market. 

In a notable development, Lunar Lake chips will include on-board memory, akin to Apple Silicon. Options of 16GB or 32GB of RAM will be available, but like Apple’s design, these will not be upgradable. 

This integration allows for a reduction in latency and a 40 percent decrease in system power usage, although it limits users needing more RAM until Intel’s next chip family, Arrow Lake, is released.

Lunar Lake will also feature eight cores, split between performance and efficiency (P-cores and E-cores). The chip includes an “advanced low-power island” for efficiently managing background tasks, contributing to a claimed 60 percent improvement in battery life over Meteor Lake.

Despite these enhancements, Intel faces competition from AMD’s Ryzen AI 300 chips, launching in July with 50 TOPS NPUs, and Qualcomm’s Snapdragon X Elite and X Plus chips. These competitors highlight the aggressive push within the AI PC market. 

Qualcomm’s chips, known for their power efficiency, reportedly achieve over 20 hours of battery life on Copilot+ Surface devices, although independent testing is pending.

Connectivity for Lunar Lake will include Wi-Fi 7, Bluetooth 5.4, PCIe Gen5, and Thunderbolt 4. However, Intel has not yet committed to integrating Thunderbolt 5, which is expected to launch later this year.

During a media briefing ahead of Computex, Intel shared benchmark results, indicating Lunar Lake’s superiority over Meteor Lake in tasks like running Stable Diffusion. Lunar Lake completed 20 iterations in 5.8 seconds, compared to 20.9 seconds for Meteor Lake, despite drawing slightly more power.

Specific chip models and deeper specifications for Lunar Lake are yet to be disclosed, but Intel’s latest offerings mark a significant stride in AI and PC performance, setting high expectations for their launch this fall.

The post Intel Finally Unveils Lunar Lake AI Chip for Copilot+ PC appeared first on AIM.

In a move to address the evolving needs of small and medium-sized enterprises (SMEs) and hosted IT service providers, AMD has introduced the AMD EPYC 4004 Series processors. 

These new offerings, announced today, complement AMD’s existing EPYC server CPU lineup, providing cost-optimised solutions without compromising on performance and enterprise-class features.

Powered by the efficient “Zen 4” architecture, the AMD EPYC 4004 Series CPUs offer enterprise-grade performance, scalability, and modern security features, catering to price-conscious buyers. 

Notably, a server equipped with a single AMD EPYC 4564P CPU outperforms an Intel Xeon E-2488 CPU by 1.8 times in terms of performance per dollar.

John Morris, corporate vice president of the Enterprise and HPC Business Group at AMD, emphasised the significance of these processors for businesses that historically had to settle for IT solutions that didn’t fully meet their requirements. “Based on the same technologies that power the most demanding data centres in the world, the AMD EPYC 4004 Series processors are offered at an optimised acquisition cost for customers in small and medium-sized businesses seeking to drive better business outcomes,” he said.

The AMD EPYC 4004 Series processors are engineered to deliver robust, general-purpose computing in a single-socket package, facilitating highly performant rack scale, multi-node, and tower configurations, particularly suitable for scenarios where system cost and infrastructure constraints are crucial considerations.

Key industry players have expressed their support for AMD’s initiative. Kamran Amini, Vice President and General Manager for Server, Storage & Software Defined Solutions at Lenovo, praised AMD’s efforts in expanding its EPYC processor roadmap to address a broader market segment with affordable yet high-performance capabilities.

OVHcloud’s chief product and technology officer, Yaniv Fdida, echoed the sentiment, expressing enthusiasm about adding AMD EPYC 4004 CPU-powered solutions to their Bare Metal portfolio, emphasising the potential for flexibility and performance-price ratio benefits in data centers.

Supermicro’s SVP Marketing and Network Security, Michael McNerney, highlighted the enhanced value brought by AMD EPYC 4004 Series CPUs to customers seeking cost-effective and easy-to-deploy solutions, particularly in workload performance optimization for hosting, content delivery, and cloud workloads.

Overall, the AMD EPYC 4004 CPU-powered servers promise a compelling balance of performance, scalability, and affordability, catering to a wide range of enterprise solutions. Supported by leading partners such as Altos, ASRock Rack, Gigabyte, MSI, New Egg, Tyan, and others, these processors signify AMD’s commitment to meeting the diverse needs of growing businesses.

The post AMD Unveils EPYC 4004 Processors to Compete with Intel’s Xeon Processors appeared first on AIM.

Intel has revealed that starting from the third quarter of 2024, its highly anticipated client processors, codenamed Lunar Lake, are slated to power over 80 fresh laptop designs across more than 20 original equipment manufacturers (OEMs). 

These processors are primed to usher in a new era of AI performance on a global scale for Copilot+ PCs, set forward by Microsoft.

Underlining the significance of this development, Michelle Johnston Holthaus, Executive Vice President and General Manager of the Client Computing Group at Intel, emphasised the breakthrough power efficiency and compatibility of the x86 architecture. “With breakthrough power efficiency, the trusted compatibility of x86 architecture and the industry’s deepest catalogue of software enablement across the CPU, GPU and NPU, we will deliver the most competitive joint client hardware and software offering in our history with Lunar Lake and Copilot+,” he said.

An AI PC, comprising a CPU, GPU, and NPU, is tailored with specific AI acceleration capabilities. The NPU, in particular, serves as a specialised accelerator for AI and machine learning tasks directly on the PC, bypassing the need for cloud processing. The rising importance of AI PCs stems from the growing necessity to automate and optimise tasks on personal computers.

Lunar Lake is anticipated to revolutionise mobile processing for AI PCs, boasting over three times the AI performance compared to its predecessors. With an impressive 40+ NPU tera operations per second (TOPS), Intel’s next-gen processors are poised to deliver the capabilities required for the upcoming Copilot+ experiences. Moreover, Lunar Lake will feature over 60 GPU TOPS, amounting to more than 100 platform TOPS.

“The launch of Lunar Lake will bring meaningful fundamental improvements across security, battery life, and more thanks to our deep co-engineering partnership with Intel. We are excited to see Lunar Lake come to market with a 40+ TOPS NPU which will deliver Microsoft’s Copilot+ experiences at scale when available,” said Pavan Davuluri, Corporate Vice President of Windows + Devices at Microsoft.

Recognising the importance of both hardware innovation and software enablement, Intel is actively collaborating with over 100 independent software vendors through its AI PC Acceleration Program. This initiative aims to enhance AI PC experiences across various domains, including personal assistants, audio effects, content creation, gaming, security, streaming, and video collaboration.

According to reports, AMD is also coming up with its new APU, Ryzen 8050, featuring Zen 5 CPU and XDNA 2 NPU architecture for AI PC workloads. This will boast a performance of around 50 TOPS, ideal for running Microsoft’s AI PC goals.

The post Intel Lunar Lake Arriving Q3 2024 with 40+ TOPS for AI PCs appeared first on AIM.

According to a recent report by IDC, unveiled at Intel’s AI for India Conference in Delhi, India’s spending on AI may reach $5.1 billion by 2027. This surge is attributed largely to AI infrastructure provisioning. This includes spending on hardware such as servers and chips, as well as software components like frameworks and libraries.

Santhosh Viswanathan, vice president and managing director, India region, Intel, said, “With an unmatched talent pool, frugal innovation, and data at scale, India stands poised to lead the global AI revolution.” He added that when it comes to building AI capabilities within India, the country does not necessarily need to rely on big GPUs.

Viswanathan said that when it comes to most of the solutions being built in India, Intel’s Xeon processors are enough to deliver the AI needs. “If you are an enterprise running a model with say 15 to 30 billion parameters, Xeon is enough to run these models effectively,” he said. 

Viswanathan also highlighted that if companies are building models for RAG on personal data inference, Xeon becomes a powerhouse. “If you have small datasets that are very local and do not have many parameters, Xeon is available everywhere for you to test and try out,” he added, saying that customers can already test out the current models available in the market on the existing Xeon-powered data centres across the country.

Xeon is omnipresent

“Not everybody is trying to build the next largest LLM and needs a trillion parameters,” said Viswanathan. Another use case that he highlights is on-edge, for which Intel’s CPU and NPU are very well positioned for privacy and the cost is significantly lower too. 

“AI is not everywhere yet, it’s in one place and you need a lot of GPUs and massive data centres [for building AI]. But over time this is going to change and the costs will come down,” he added.

“You do not need to go back and build massive infrastructures. AI can start today with the infrastructure that you have,” he said. Viswanathan explained that Intel’s go-to-market strategy is about making customers in India realise that they can existing infrastructure that is already using Xeon processors.

Viswanathan said that the reason Intel is going bullish on India is the country’s ability to solve big problems with frugality, like in the case of UPI. He narrated how Intel was the company to bring WiFi in India and just like the internet, Viswanathan said, Intel wants to bring AI everywhere in India. 

“Intel’s goal is to democratise access, and the architecture is open,” said Viswanathan. He added that today, people are waiting for compute and this is where Intel comes in with its Xeon processors. Apart from running high-end AI models, Xeon is also effective and scalable for other workloads, and does not cost as much. 

“That is why I am bullish on Xeon as it is already available across all databases. It is omnipresent,” he added.

Intel also offers its Developer Cloud where customers can test out its offerings while running them in a secure environment. 

For Intel, AI stands for ‘Amazing India’

“When you really need to build something big and test the performance, Gaudi is always there,” Viswanathan said, and added that the company is working with several partners in India to test and benchmark its AI hardware. All of this is along with making AI PCs in partnership with OEM ecosystems such as HPE, Dell, and Lenovo. 

Furthermore, the recently announced Gaudi 3 at Intel Vision accelerator is expected to outperform the NVIDIA H100 by 50% in inference throughput on an average and achieve a 40% increase in inference power-efficiency across different parameter models. 

This, along with the newer Xeon 6 processor, are also optimised heavily for RAG.

Intel is positioning itself in the market as a low-cost alternative to its competitors like NVIDIA and AMD. Viswanathan said that Intel is always an alternative for a company that is struggling with acquiring compute as the cost is too high. He explains that Xeon is a workhorse for a lot of use cases that do not need an accelerator.

Intel indeed has been bullish on India. There were several collaborations announced at the Intel Vision 2024 such as Bharti Airtel, Infosys, and Ola Krutrim. Moreover, Zoho is also leveraging Intel’s processors for its generative AI offerings.

Infosys’ partnership with Intel is about integrating 4th and 5th Gen Intel Xeon processors, Intel Gaudi 2 AI accelerators, and Intel Core Ultra into Infosys Topaz. This collaboration aims to offer AI-first services, solutions, and platforms to accelerate business value through generative AI technologies. 

Ola Krutrim recently launched its open-source model on Databricks platforms. The company utilised Intel Gaudi 2 clusters to pre-train and fine-tune its foundational models with generative capabilities in ten languages, achieving industry-leading price/performance ratios compared to existing market solutions. 

Additionally, Krutrim is currently pre-training a larger foundational model on an Intel Gaudi 2 cluster, further advancing its AI capabilities.

Intel also has Make in India partners and is in talks with the government to build systems locally and fully designed in India. “Anybody who is keen on reducing the carbon footprint while also reducing the cost on their wallet, we are absolutely there,” added Viswanathan.

The post Intel is Bullish on India with its Xeon Processors appeared first on AIM.

Intel has unveiled the world’s largest neuromorphic system, named Hala Point, to promote more sustainable and efficient AI. Utilising Intel’s Loihi 2 processor, the system is designed for research in brain-inspired AI and addresses challenges in today’s AI efficiency and sustainability. 

Hala Point, initially deployed at Sandia National Laboratories, improves on Intel’s previous large-scale research system, Pohoiki Springs, with over 10 times more neuron capacity and up to 12 times higher performance.

“The computing cost of today’s AI models is rising at unsustainable rates. The industry needs fundamentally new approaches capable of scaling. For that reason, we developed Hala Point, which combines deep learning efficiency with novel brain-inspired learning and optimisation capabilities. We hope that research with Hala Point will advance the efficiency and adaptability of large-scale AI technology,” said Mike Davies, director of the Neuromorphic Computing Lab at Intel Labs.

Hala point can support up to 20 quadrillion operations per second, or 20 petaops, with an efficiency exceeding 15 trillion 8-bit operations per second per watt (TOPS/W) when executing conventional deep neural networks.

These capabilities surpass those of systems based on GPUs and CPUs. Its advanced features enable real-time continuous learning for applications such as smart city management, scientific problem-solving, and large language models.

Sandia National Laboratories plans to use Hala Point for advanced brain-scale computing research, focusing on scientific computing challenges across various disciplines. The system’s large-scale capacity allows researchers to tackle complex problems in fields ranging from commercial to defence to basic science.

“Working with Hala Point improves our Sandia team’s capability to solve computational and scientific modelling problems. Conducting research with a system of this size will allow us to keep pace with AI’s evolution in fields ranging from commercial to defence to basic science,” said Craig Vineyard, Hala Point Team Lead, Sandia National Laboratories.

While Hala Point is a research prototype, its development promises advancements such as continuous learning in large language models, significantly reducing the training burden in AI deployments. Intel anticipates further progress in the field by applying neuroscience-inspired computing principles to minimise power consumption and maximise performance.

The post Intel Builds Largest Neuromorphic System for Sustainable AI appeared first on AIM.

Indian companies are very much dedicated to building their own AI models. And Intel has been a long lover when it comes to delivering solutions within the country in every technological domain. Now, Intel is taking another step forward by partnering with Indian companies for delivering its AI hardware. 

“Boy was I startled when I learned that India is very convinced they need their own models for their environment” said Patrick Gelsinger, CEO of Intel at Intel Vision 2024. “They are excited to train and be able to deliver that using Gaudi clusters,” he added. 

Apart from Xeon 6 and Gaudi 3, there were various new collaborations announced at the event, many within India. 

The buzzing partner ecosystem

Bharti Airtel aims to harness its extensive telecom data to enhance AI capabilities, thereby enriching customer experiences and exploring new revenue avenues in the digital realm.

Infosys has announced a strategic partnership with Intel, integrating Intel technologies such as 4th and 5th Gen Intel Xeon processors, Intel Gaudi 2 AI accelerators, and Intel Core Ultra into Infosys Topaz. This collaboration aims to offer AI-first services, solutions, and platforms to accelerate business value through generative AI technologies.

Infosys also plans to utilise Intel’s AI training resources to educate its employees about Intel’s product offerings, enabling them to offer generative AI expertise to the company’s extensive international customer base across various industries.

Ola Krutrim is utilising Intel Gaudi 2 clusters to pre-train and fine-tune its foundational models with generative capabilities in ten languages, achieving industry-leading price/performance ratios compared to existing market solutions. Additionally, Krutrim is currently pre-training a larger foundational model on an Intel Gaudi 2 cluster, further advancing its AI capabilities.

CtrlS, one of the largest and fastest growing data centres in the world, which is hosting most of the providers in India is also using Gaudi 2 and Xeon processors, revealed Gelsinger in his keynote. 

In March, L&T had also announced its collaboration with Intel for deploying scalable edge-AI solutions across various domains, including Cellular Vehicle-to-Everything (CV2X) applications, leveraging the expertise in connected vehicles and smart transportation systems alongside Intel’s Edge Platform.

Just this month, Zoho also collaborated with Intel for optimising AI workloads within the company. Santosh Viswanathan, vice president and managing director at Intel, said that Zoho has witnessed significant performance improvements in AI workloads with 4th Gen Intel Xeon processors.

Though most of these partnerships come with the last generation of Gaudi and Xeon, the leadership has been quite vocal about the expansion plans within the country.

India is set as a distinct entity

Intel is betting big on India, which is not that new. Providing cheaper alternatives when it comes to data centres and powering enterprise solutions, Intel has always been the go to choice for Indian companies. While NVIDIA is increasingly expanding its partnership within India with entities like Yotta for establishing data centres, Intel remains a viable option for already established customers within India. 

“AI does not just require big GPUs to solve the problem. There are a lot of different models that can run on Xeon. Innovation at scale can happen with Xeon. We are working with several large customers. Gaudi 2 is available, Gaudi 3 comes in the second half. You will see some of those products coming into India through these customers as well,” Viswanathan said earlier.

Christoph Schell, executive vice president of Intel said that the company is betting big on India when it comes to AI by carving it out as a separate geographic region. The American chip manufacturer is introducing a new era of computing with the release of its AI-powered PC in late 2023. 

These systems, featuring Intel’s Core Ultra processors tailored for AI tasks, improve user productivity and experience. Intel’s AI PCs are currently available in the market, and numerous retailers have started distributing them in India. By 2025, Intel aims to supply core processors for as many as 100 million AI-enabled PCs, much of which would be through India. 

Though Viswanathan has said that the company currently has no plans for setting up its fab within the country, it is still betting big on AI in India through other ways. 

“World needs a balanced supply chain. You cannot have 80% of servers being made in one place and 90% of all laptops made in one place. I think that’s the key change where India can really step and help build a balanced electronics supply chain for the world,” he said.

Viswanathan said India has about 20% of the world’s data sets that can be used for AI models training.

“We are very frugal. 16 or 20% of the world’s AI talent is in India. We kind of lead the world and not follow in this path. That’s another piece that makes me bullish about India. For me India is most exciting. AI is not just artificial intelligence, it is also amazing in India. No other country has digital infrastructure at the scale that we have. India stack is a game changer,” Viswanathan said.

The post India is a Sweet Spot for Intel appeared first on AIM.

At the Intel Vision 2024, the company has introduced its latest innovation in the realm of data centre, cloud, and edge computing with the launch of the new Intel Xeon 6 processors. 

Designed to offer performance-efficient solutions for running AI applications such as RAG, these processors aim to deliver business-specific results by leveraging proprietary data. The new brand, Intel Xeon 6, heralds a significant leap in processing power and efficiency, catering to the evolving needs of modern computing landscapes.

The current ‘Emerald Rapids’ Fifth-Gen Xeon models from Intel will not undergo a rebranding, indicating that the new branding scheme will exclusively pertain to Xeon 6 and subsequent processor iterations.

Under the hood, the Intel Xeon 6 processors boast two distinct variants: those equipped with Efficient-cores (E-cores) and those featuring Performance-cores (P-cores). 

The E-core processors, codenamed Sierra Forest, promise a remarkable 2.4x improvement in performance per watt and a staggering 2.7x enhancement in rack density compared to their predecessors, the 2nd Gen Intel Xeon processors. This advancement not only amplifies computational capabilities but also enables customers to replace outdated systems at a ratio of nearly 3-to-1, thereby substantially reducing energy consumption and contributing to sustainability goals.

On the other hand, the P-core processors, codenamed Granite Rapids, introduce software support for the MXFP4 data format. This integration results in a notable reduction in next token latency by up to 6.5x compared to the 4th Gen Intel Xeon processors using FP16. Furthermore, with the ability to run 70 billion parameter Llama 2 models, these processors are poised to elevate AI performance to unprecedented heights.

At the Intel AI Everywhere event in December, Intel had revealed the forthcoming release of 5th Gen Xeon processors, featuring AI acceleration in every core and expected to hit the market in 2024. Unveiled by Intel CEO Pat Gelsinger, these processors, previously codenamed Emerald Rapids, mark a significant advancement in computing. 

In addition to advancements in processing power, Intel has also announced significant developments in client, edge, and connectivity solutions. The company’s Intel Core Ultra processors are driving new capabilities for productivity, security, and content creation, presenting an enticing proposition for businesses to refresh their PC fleets. 

Intel anticipates shipping 40 million AI PCs in 2024, featuring over 230 designs spanning from ultra-thin PCs to handheld gaming devices.

Looking ahead, Intel’s roadmap includes the launch of the next-generation Intel Core Ultra client processor family, codenamed Lunar Lake, in 2024. This lineup is projected to deliver more than 100 platform tera operations per second (TOPS) and over 45 neural processing unit (NPU) TOPS, ushering in a new era of AI-centric computing.

Furthermore, Intel has unveiled new edge silicon across its product families, targeting key markets such as retail, industrial manufacturing, and healthcare. These additions to Intel’s edge AI portfolio are slated for availability this quarter and will be supported by the Intel Tiber Edge Platform throughout the year.

In a bid to revolutionise Ethernet networking for AI fabrics, Intel is spearheading the Ultra Ethernet Consortium (UEC), introducing a range of AI-optimised Ethernet solutions. These innovations are designed to cater to the evolving needs of large-scale AI fabrics, enabling seamless training and inferencing for increasingly complex models.

The post Intel Unveils Xeon 6 Processors appeared first on AIM.

Intel has announced its latest AI chip Gaudi 3 at the Intel Vision 2024 event, in a bid to keep pace with the growing demand for semiconductors capable of training and deploying large AI models.

The newly introduced Gaudi 3 chip, which was revealed by CEO Pat Gelsinger at Intel AI Everywhere event, boasts over double the power efficiency compared to its predecessor and is capable of running AI models 1.5 times faster than NVIDIA’s H100 GPU. 

It offers various configurations, including a bundle of eight Gaudi 3 chips on one motherboard or a card that can be integrated into existing systems.

Gaudi 3, built on a 5 nm process, signals Intel’s utilisation of advanced manufacturing techniques. Additionally, Intel plans to manufacture AI chips, potentially for external companies, at a new Ohio factory expected to open in the coming years, according to Gelsinger.

During testing, Intel evaluated the chip’s performance on models like Meta’s open-source Llama and the Falcon model by TII. Gaudi 3 demonstrated its capability to facilitate the training or deployment of models such as Stable Diffusion or OpenAI’s Whisper model for speech recognition.

Compared to the NVIDIA H100, Intel anticipates that Gaudi 3 will achieve an approximately 50% faster time-to-train on average across Llama 2 models with 7B and 13B parameters, as well as the GPT-3 175B parameter model. 

While performance data for NVIDIA’s recently announced Blackwell-based B200 Tensor GPU is not currently available, it’s clear that NVIDIA’s latest offering would likely affect these performance comparisons significantly.

In comparison to NVIDIA, Intel claims its chips consume less power. NVIDIA currently dominates approximately 80% of the AI chip market with its GPUs, which have been the preferred choice for AI developers in the past year.

Intel asserts that its Gaudi 3 AI accelerator offers an estimated 50% enhancement in inferencing performance and around 40% better power efficiency compared to NVIDIA’s H100. Moreover, Intel states that it achieves these benefits at a significantly lower cost.

Intel has announced that Gaudi 3 chips will be available to customers in the third quarter, with companies like Dell, HPE, and Supermicro set to incorporate the chips into their systems. However, Intel hasn’t disclosed the pricing details for Gaudi 3.

Das Kamhout, vice president of Xeon software at Intel, expressed confidence in Gaudi 3’s competitiveness against NVIDIA’s latest offerings, citing factors such as competitive pricing and the incorporation of an open integrated network on chip.

The data centre AI market is expected to expand as cloud providers and businesses invest in infrastructure for deploying AI software, indicating opportunities for other players in the market.

While NVIDIA has seen significant stock growth driven by the AI boom, Intel’s stock has experienced more modest gains. Nevertheless, Intel remains determined to compete in the AI chip market, with AMD also seeking to expand its presence in the server AI chip segment.

NVIDIA’s success has largely been attributed to its proprietary software suite, CUDA. In contrast, Intel is collaborating with chip and software giants like Google, Qualcomm, and Arm to develop open software solutions, aiming to provide greater flexibility for software companies in selecting chip providers.

In addition, Intel unveiled its intention to create an open platform for enterprise AI, aiming to expedite the deployment of secure GenAI systems empowered by retrieval augmented generation (RAG).

The post Intel Unveils the Most Efficient Gaudi 3 AI Accelerator at Intel Vision appeared first on AIM.

In a recent blog post titled “Behind the Compute,” Stability AI, unveiled shocking findings regarding the performance of Intel Gaudi 2 accelerators compared to NVIDIA’s H100 in training and inference of its upcoming image generation model Stable Diffusion 3.

Stability AI’s text-to-image model demonstrated promising results in the performance analysis. Utilising the 2B parameter multimodal diffusion transformer (MMDiT) version of the model, Stability AI compared the training speed of Intel Gaudi 2 accelerators with NVIDIA’s A100 and H100.

On 2 nodes configuration, Intel Gaudi 2 system processed 927 training images per second, 1.5 times faster than NVIDIA H100-80GB. Further increasing the batch size to 32 per accelerator in Gaudi 2 resulted in a training rate of 1,254 images/sec.

On 32 Nodes Configuration, the Gaudi 2 cluster processed over 3x more images per second compared to NVIDIA A100-80GB GPUs, despite A100s having a highly optimised software stack.

On inference tests with the Stable Diffusion 3 8B parameter model, Gaudi 2 chips offered similar inference speed to NVIDIA A100 chips using base PyTorch.

However, Stability AI admitted that with TensorRT optimisation, A100 chips produced images 40% faster than Gaudi 2, but Stability AI anticipates Gaudi 2 to outperform A100s with further optimisation. This can be further contented with the upcoming GH200 processors that might be announced at GTC 2024 this month. 

Few months back, AMD also claimed that it has surpassed NVIDIA H100 on various performance metrics, but it was later debunked by NVIDIA as it said that AMD also did not include TensorRT optimisation for the test. 

Intel has also launched its Gaudi 3 AI accelerator which would make this competition even interesting in the future. 

Moreover, Stable Beluga 2.5 70B, Stability AI’s fine-tuned version of LLaMA 2 70B, showcased impressive performance on Intel Gaudi 2 accelerators. Running the PyTorch code out of the box on 256 Gaudi 2 accelerators, Stability AI measured an average throughput of 116,777 tokens/second.

Gaudi 2 demonstrated a 28% faster performance compared to NVIDIA A100 in inference tests with the 70B language model, generating 673 tokens/second per accelerator.

The post Stability AI Claims Intel Gaudi 2 is Faster than NVIDIA H100 appeared first on AIM.

In a significant move to enhance smart city infrastructure and intelligent transportation systems, L&T, a leading global digital engineering and R&D services company, has announced a collaboration with Intel Corporation. 

This partnership is set to develop and deploy scalable edge-AI solutions across various domains, including Cellular Vehicle-to-Everything (CV2X) applications, leveraging LTTS’s expertise in connected vehicles and smart transportation systems alongside Intel’s cutting-edge Edge Platform.

The collaboration aims to harness the power of Intel’s Edge Platform, equipped with built-in AI runtime and OpenVINO inference for real-time AI inferencing optimisation. This technology will enable LTTS to enhance traffic management and emergency safety measures in smart cities and transportation sectors, addressing the critical need for advanced networking and AI analytics at the edge with low latency, locality, and cost-effectiveness.

Abhishek Sinha, Chief Operating Officer and Board Member at L&T Technology Services, expressed enthusiasm about the partnership, stating, “LTTS is delighted to collaborate with Intel on launching their new Edge Platform, which promises to democratise access to edge-AI solutions. With deep-rooted hardware optimisation at its core, our enterprise customers can trust Intel’s Edge Platform to propel them into a future of unparalleled performance and possibilities.”

Intel’s Edge Platform heralded as a game-changer, offers a comprehensive ecosystem with modular building blocks and premium service and support offerings, designed to scale infrastructure across various industries horizontally.

Pallavi Mahajan, Intel Corporate Vice President And General Manager of Network and Edge Group Software highlighted the benefits of this collaboration for industries such as transportation and smart cities. She noted, “The collaboration with LTTS on Intel’s Edge Platform will simplify the exchange of critical information and streamline infrastructure management to improve results and lower TCO for customers.”

This partnership underscores LTTS’s commitment to advancing smart cities and road infrastructure, setting the stage for developing intelligent transportation systems that promise enhanced road safety, accident prevention, and improved mobility for the future.

The post L&T and Intel Partner to Advance Edge-AI Solutions to Enhance Smart Cities appeared first on AIM.

HCLTech and Intel Foundry have announced their decision to expand their collaboration to co-develop silicon solutions to improve semiconductor innovation globally. This partnership will leverage HCLTech’s design expertise and Intel Foundry’s advanced technology and manufacturing capabilities. 

The goal is to establish a resilient and diversified supply chain to meet the rising global demand for semiconductor manufacturing. This collaboration will offer semiconductor manufacturers, system OEMs, and cloud services providers a robust ecosystem for semiconductor sourcing. Additionally, the collaboration has the potential to spur innovation by enabling the design of customised silicon solutions tailored to specific use cases.

“Intel Foundry’s advanced technologies and silicon-verified IPs in manufacturing and advanced packaging strengthens our delivery of innovative, accessible and diverse solutions to our mutual clients. This will also give them greater choice and flexibility in semiconductor sourcing,” said Vijay Guntur, President, Engineering and R&D Services, HCLTech.D

HCLTech has been collaborating with Intel for over 30 years, a relationship that has evolved through shared offerings and joint investments in various sectors, including silicon services, hardware engineering, telecom services, and more. The current focus is on jointly designing highly customised silicon solutions for companies, combining HCLTech’s design expertise with Intel’s manufacturing capabilities.

This expanded collaboration is set to further strengthen their partnership by fostering a strong and open ecosystem beneficial for clients requiring advanced silicon solutions.

Intel also announced that it has signed Microsoft as a foundry customer for a custom chip. This deal is part of Intel’s plan to overtake TSMC using its Intel 18A and upcoming 14A manufacturing technologies. The 18A, a 1.8nm technology, is set for early 2025 and will be used for future CPUs in both consumer and data centre markets. The 14A, planned for late 2026, will introduce a more advanced lithography tool for smaller and more efficient chips. 

Together with its collaboration with HCLTech to develop customised silicon solutions, Intel aims to meet the growing demand for semiconductors. 

The post Intel Collaborates with HCLTech to Advance Semiconductor Manufacturing appeared first on AIM.

Intel Foundry Services (IFS) is set to host IFS Direct Connect on February 21, 2024, at the San Jose McEnery Convention Center. Sam Altman, Chief Executive of OpenAI, will mark his presence at the event as a luminary speaker.

“Thrilled Sam Altman is joining me at Direct Connect on Feb 21. Sam is a renowned leader on AI & its impact on the world. Looking forward to discussing the role semis play in enabling modern society,” wrote Pat Gelsinger, Intel chief.

“Infinite possibilities ahead in the age of AI & no better convo for Intel’s 1st Foundry event. See you soon, Sam!” he added.”

The dialogue is expected to delve into the critical role played by semiconductors in shaping modern society, exploring the limitless possibilities in the age of AI.The IFS Direct Connect event promises exclusive insights from Intel executives and ecosystem partners. 

Attendees can anticipate learning about Intel’s strategies, process technology, advanced packaging, and ecosystem collaborations. Of particular interest will be the discussion on how Intel Foundry Services can empower silicon designs, leveraging Intel’s resilient, secure, and sustainable supply chain.

Recently, reports surfaced that Altman is planning to raise billions for an AI chip venture aimed at developing a ‘network of factories’ for fabrication that would stretch around the globe, involving collaboration with unnamed ‘top chip manufacturers.’ 

Last week, Altman visited South Korea to meet with executives from Samsung Electronics and SK Hynix, exploring the potential for an alliance in the field of AI chips. Other reports suggest that Altman is in discussions with Middle Eastern investors and chip fabricators, including TSMC, about launching a new chip venture.

The post Sam Altman to Attend  Intel’s Event, Indicates Venturing into AI Chips appeared first on AIM.

Recently, Intel researchers unveiled a new LLM inference solution with low latency and high throughput for Intel GPUs. They showed that their solution achieved up to 7x lower latency and up to 27x higher throughput than standard HuggingFace implementation. 

As LLMs continue to play a pivotal role across various industries, optimising their performance has become a critical focus, and Intel’s latest development promises to be a game-changer. Tackling the inherent complexity of LLMs, characterised by intricate model structures and autoregressive inference modes, the team behind this breakthrough presents an efficient alternative.

One of the primary challenges the research team addresses is the intricate design of LLMs, characterised by intricate model structures and extensive autoregressive operations. The complexity leads to massive memory access and hampers inference speed.

A simplified LLM decoder layer is at the heart of their solution, strategically designed to fuse data movement and element-wise operations. This fusion reduces memory access frequency and significantly lowers system latency, paving the way for faster and more efficient inference processes.

Read: What is Intel’s AI Plan for 2024

How is Intel pushing the boundaries?

Intel’s solution begins with a streamlined approach to the LLM decoder layer. The team successfully reduces memory access frequency by fusing data movement and element-wise operations, substantially lowering system latency.

Another key innovation is introducing a segment KV (key/value) cache policy. This strategic separation of key and value elements for request and response tokens in distinct physical memory segments proves instrumental in effective device memory management. The outcome is an expanded runtime batch size and improved overall system throughput.

The team customises a Scaled-Dot-Product-Attention kernel to complement their innovative approach, aligning it seamlessly with their fusion policy based on the segment KV cache solution. The result is a finely tuned LLM inference solution that promises to reshape the efficiency standards for these powerful language models.

The research team has not only conceptualised these innovations but has also translated them into a practical solution. Their LLM inference solution is implemented on Intel GPUs and is now publicly available for scrutiny and use.

The substantial reduction in token latency enhances system responsiveness, making it an ideal fit for applications where quick processing is crucial. Simultaneously, the significant boost in throughput facilitates the swift execution of larger tasks, making this solution particularly attractive for real-world, high-demand scenarios.

The post Intel Unveils New Low-Latency LLM Inference Solution Optimized for Intel GPUs appeared first on AIM.

Intel in the beginning of its generative AI focused year, has revealed the creation of a new platform company named Articul8 AI. Backed by Boca Raton, Florida-based asset manager and investor DigitalBridge, the initiative stems from a proof-of-concept developed through Intel’s collaboration with Boston Consulting Group (BCG) in May of the previous year.

Intel utilised its hardware along with a combination of open-source and internally sourced software to craft a generative AI system capable of processing text and images. The system, initially designed over a two-year period within Intel, was further fine-tuned to meet BCG’s specific security requirements.

Read: What is Intel’s AI Plan for 2024

While BCG initially served as the exclusive supplier and customer of the system, Intel has been actively working to scale the platform over the past few months. The platform, optimised for Intel hardware but compatible with alternatives, is targeted at industries such as financial services, aerospace, semiconductor, telecommunications, and others with high-security and specialised domain knowledge needs.

“With its deep AI and HPC domain knowledge and enterprise-grade GenAI deployments, Articul8 is well positioned to deliver tangible business outcomes for Intel and our broader ecosystem of customers and partners. As Intel accelerates AI everywhere, we look forward to our continued collaboration with Articul8,” said Pat Gelsinger, Intel CEO.

Arun Subramaniyan, formerly a VP and GM at Intel’s data centre and AI group, has been appointed as the CEO of Articul8. The spinout will consist mainly of ex-Intel employees, and Intel will retain an undisclosed stake in the newly formed company.

Now, Justin Hotard will be the executive vice president and general manager of Intel’s Data Center and AI Group.

Apart from Intel and DigitalBridge, Articul8’s investors include Fin Capital, Mindset Ventures, Communitas Capital, GiantLeap Capital, GS Futures, and Zain Group.

According to a spokesperson, Intel and Articul8 will remain strategically aligned, with plans to leverage Articul8’s enterprise gen AI software for internal use cases and as part of a joint go-to-market partnership with end customers. 

This move is part of Intel’s broader strategy to seek outside capital for business units, following previous spinouts such as Mobileye and the sale of its memory chip division. The company also intends an eventual initial public offering of its programmable chip unit. These strategic manoeuvres align with Gelsinger’s comeback plan, involving the expansion of chip factories in the U.S. and Europe and the introduction of advanced chip manufacturing nodes within the next four years.

The post Intel Launches Articul8 AI, an Enterprise Generative AI Company appeared first on AIM.

Intel has a well thought out plan for the upcoming years when it comes to AI. And it was pretty evident at the recent AI Everywhere event hosted by the company, where the company announced a line of products like AI PCs, five semiconductor nodes, and the upcoming Gaudi3 AI accelerator. 

“Shortly after my return to Intel Corporation, we set out a tremendously audacious goal – to manufacture five new process technology nodes in just four years,” Pat Gelsinger, CEO of Intel posted on LinkedIn.

Two nodes, Intel & and Intel Core Ultra were announced at the AI Everywhere event. While Intel 3 is going into production next year, Intel 20A which he calls a work of art, and the last 18A will be available by the end of next year and are developing in the fab. Gelsinger posed with them like a happy family. 

“I like to just have one other little thing to show off here and they just brought it out of the lab,” just before the end of the conference, Gelsinger walked in with the first ever Gaudi3 AI accelerator.

“When we first announced this goal, many stated this would be nearly impossible. But we’re doing what we set out to do – a decade of semiconductor work in just four years. Two out of these five nodes are complete. The remaining three are underway and on-track.” Intel’s roadmap for AI looks great and a lot of it is already underway. 

Gelsinger believes that the supercomputer that Intel is building will be the largest in Europe, powered by Gaudi3. Intel also announced the 5th generation of Xeon processors that would power Microsoft, Google, and IBM’s data centres. 

The expansionist agenda

To ensure that the roadmap works well, Intel has secured a $3.2 billion grant from the Israeli government for the construction of a new $25 billion chip plant in southern Israel. This marks the largest investment ever made by a company in Israel. Intel’s expansion plan at its Kiryat Gat site, located 42 km from Hamas-controlled Gaza, is described by Intel as a crucial step in fostering a more resilient global supply chain. 

Gelsinger has projected that the GPU market size would be around $400 billion by 2027. This definitely gives room for a lot of competitions to thrive, and thus there is a lot expected from Intel. Thus, he has led Intel’s substantial investments in building chip factories across three continents to regain dominance in chip manufacturing.

In Germany, Intel is set to invest over 30 billion euros ($33 billion) in establishing two chip-making plants in Magdeburg, as part of a multi-billion-dollar investment initiative in Europe to boost chip production. Germany has offered substantial subsidies to secure its largest-ever foreign investment. 

Additionally, Intel also announced a plan in 2022 to invest up to $100 billion in building what could be the world’s largest chip-making complex in Ohio.

Intel also has a bunch of partnerships for its AI PCs goals. At the event, Gelsinger and his team announced the launch of Intel Core Ultra and Intel Arc GPUs for pushing the goal of making every PC in the world an AI PC. This will be achieved by its partnership with Dell, Lenovo, HP, Supermicro, and Microsoft, for bringing the hardware onto their devices. 

Intel’s unique position in the industry, marked by openness, scalability, and end-to-end solutions, allows for the seamless infusion of AI capabilities into every platform. This is well established given that Intel was the first to produce the first commercial microprocessor chip in 1971, and has been the leader in developing components for PCs.

A long roadmap

During Intel’s recent conference call, Gelsinger stated, “Our Gaudi roadmap remains on track with Gaudi3 out of the fab, now in packaging and expected to launch next year. Looking ahead to 2025, Falcon Shores will integrate our GPU and Gaudi capabilities into a unified product.”

Gaudi3 is expected to arrive with a 5nm chip. The accelerators are set to provide a significant boost with up to 4 times the BFloat16 capabilities, double the compute power, 1.5 times the network bandwidth, and a 1.5 times increase in HBM capacities (144 GB compared to 96 GB). 

In 2025, the successor to Gaudi3, Falcon Shores, will merge the AI capabilities of Gaudi with the powerful GPUs from Intel, all within a single package. This is something that would give Intel the edge over others.

Intel is also planning to onboard another version of the AI accelerator superchip, Falcon Shores 2, by 2026, which would be based on the Gaudi3 architecture. “We have a simplified roadmap as we bring together our GPU and accelerators into a single offering,” Gelsinger said. Though this is a far-out vision, the reveal at the AI Everywhere conference also gives out some hope for the company.

Intel has a long road ahead to compete with NVIDIA and AMD, and Gelsinger is definitely helping the company pave the path faster.

The post What is Intel’s AI Plan for 2024 appeared first on AIM.

Apple surely knows how to develop amazing products and thrives on its ever-evolving ecosystem. But, what many don’t know is, it’s a trap – the ‘Apple Trap’ – for once you’re in, you are a happy captive.  

The tech giant recently released a new software update, iOS 17.2. With this, iPhone15 Pro and iPhone 15 Pro Max users can now record spatial videos, which can be brought to life with an Apple Vision Pro, which is expected to be released early next year. 

Experiencing spatial videos on Apple Vision Pro. Source: Apple Blog

In a Walled Garden

It’s not new that Apple has always pushed for a closed ecosystem. It serves as a way to not only give its users products exclusive to the club, but to retain those users through new products that work best when all Apple products are owned. With the Continuity feature, one can use their Mac with other Apple devices, allowing for smarter work and seamless transitions between the devices. 

The first Apple wearable, Apple watch, which was released in 2015, continues to allow sync only via iPhones. At the time of launch, though the iOS market was only 29%, when Android had over 70%, Apple continued its philosophy of launching it only for their niche club. Similarly, the easy pairing and content sync within iPhones and Macbooks.

Similarly, Apple TV that operates on tvOS, the operating system developed by Apple, includes related products such as Siri Remote and an Apple TV+ subscription. Clubbing this with other home devices such as Apple Pods, the whole home setup is addressed. 

Push Towards Hardware

As per a recent Bloomberg report, Apple is said to heavily concentrate on its wearable segment in 2024. The iPhone, which has always been at the center of annual Apple product launches, will also see the launch of a new iPhone version, however, it will have no significant upgrades. Instead, Apple will be focusing on their hardware products this year. 

In addition to Vision Pro, the most-anticipated Apple product for the coming year, the company will be releasing advanced versions of AirPods and watches. The healthcare industry, which has been one of the sought-after categories that Apple is now chasing, will be the main focus for the product launches. 

The company has announced plans to add more health detection features in their next series of watches, particularly hypertension and apnea among others. The Apple AirPods that also assist the hearing-disabled, will likely have more advanced features.

Apple’s new patent shows AirPods with brain wave-detecting sensors that could measure brain activity, muscle movement, blood volume, and more features. It is a given that these features will require an iPhone for use, and the way the company will integrate it across other devices is to be seen.

AI, rather ML, For its Users

Being adamant on referring to machine learning and not ‘AI’ while announcing Apple hardware updates, Apple is going big on building an ecosystem in the PC category too. Moving away from wearables, Apple announced a string of next-generation chips for Macbook on the eve of Halloween. The M3, M3 Pro and M3 Max built with 3-nanometer process technology with an improved neural engine, will boost high-performace ML models, at improved speed and efficiency. 

By announcing chips, Apple is going head-to-head against Intel, AMD and Qualcomm. With the chip offerings, Apple is trying its best to capture the chip market which is currently dominated by Intel with 68.4% followed by AMD with 31.6%. In 2020, Apple parted ways with Intel after 15 years when they launched their M1 chip, to power their MacBooks, indicating Apple’s long-term vision of non-reliability from any potential competitor.  

Apple is also building its own internal chatbot, Apple GPT, on its framework Ajax, and is also working towards bringing generative AI features into its voice-assisted platform Siri and other Mac products. The move is further fortifying Apple’s closed ecosystem, and pushing for more user adoption.

When other companies such as OpenAI and Microsoft are way ahead on AI developments, Apple’s presence in hardware and software gives it an advantage, thereby, locking-in its customers forever. 

The post Apple Smoothly Crafts ‘Mouse Traps’ for Humans appeared first on AIM.

“What do you do during the holidays?” asked Pat Gelsinger, CEO of Intel at the AI Everywhere event.

“You all gather around the tree and take a family photo. So kids, papa’s here,” said Gelsinger while posing with the upcoming nodes, with Intel 7, Intel Core Ultra, Intel 3 which is going into production next year, Intel 20A which he calls a work of art, and the last 18A, which would be available by the end of next year and developing in the fab.

Gelsinger, with all his energy, starts the event at the Big Apple, New York City. “I don’t know if you have heard about this thing called AI, but boy, this year” has been all about it. 

Intel aims to deliver 100 million client processors, which is five times more than any of the competitors. “We think 2024 marks the era of AI PC and that will be the star of the show in this coming year,” said Gelsinger. During the presentation, a slide showcased the company’s partnership with almost all the OEM providers, and the clients say that the processors are working tremendously.

Augmented Intelligence (AI) everywhere

At the event, Gelsinger and his team announced the launch of Intel Core Ultra and Intel Arc GPUs for pushing the goal of making every PC in the world an AI PC. This will be achieved by its partnership with Dell, Lenovo, HP, Supermicro, and Microsoft, for bringing the hardware onto their devices. 

“When we think about artificial intelligence, we think of something we may not understand or may not control,” said Gelsinger, “We think it’s a disservice to think about it that way. Instead, maybe augmented intelligence, how we integrate it into our human lives and to human intelligence, how we make it part of us in everything that we do. And we think that bringing this value into the human experience is the opportunity for AI”

He said this narrating a personal story, where he emphasised the transformative power of AI-enhanced Starkey Hearing aids which changed his perspective entirely. “These are making me better. It’s not something over there. It’s right here augmenting my human experience.”

AI PCs everywhere

The AI PC concept aims to bring the same vision by bringing AI capabilities directly to personal computers, allowing users to run generative AI chatbots, like ChatGPT, locally, without relying on cloud data centres, and even for inference models such as Llama 2.

This development is particularly significant for enhancing data security, a crucial aspect for business and enterprise users. But more than that, the AI PC concept is about bringing AI to everyone. This was also highlighted at AMD’s Advancing AI event with the launch of Ryzen AI PCs for the exact same purpose.

Furthermore, he announced the launch of the 5th generation of Xeon processors, for powering the data centres of its partners, which includes major cloud providers such as IBM, Google, and Microsoft. 

Why Intel?

“Are we going to dedicate a third and a half of all of the Earth’s energy to these computing technologies? No, they must be sustainable, as well,” Gelsinger discusses the convergence of high-performance computing and AI, emphasising sustainability as a crucial factor. 

Intel’s unique position in the industry, marked by openness, scalability, and end-to-end solutions, allows for the seamless infusion of AI capabilities into every platform. This is well established given that Intel was the first to produce the first commercial microprocessor chip in 1971, and has been the leader in developing components for PCs.

Gelsinger added that Intel is dedicated to developing technologies that enable seamless integration and efficient operation of AI across various applications, both in the cloud and increasingly on local devices such as PCs and edge devices.

“It takes a village of an ecosystem to build to make that happen,” said Christoph Schell, Executive Vice President at Intel, highlighting how the company has been investing in edge computing since 2018 with software like oneAPI and OpenVINO, and now has around 90,000 edge deployment applications. 

“Why do Intel customers like us? They like our open approach,” said Schell, touching upon data, technology stack, engineering, and most importantly, the ecosystem. “They like that we’re predictable.”

“I like to just have one other little thing to show off here and they just brought it out of the lab,” walked in Gelsinger with the first ever Gaudi3 processor. “They are not just on PowerPoints, They are real,” concluded Gelsinger and urged everyone to go do Christmas shopping.

The post “Kids, Papa’s Here,” Says Intel Chief Pat Gelsinger appeared first on AIM.

At the Intel AI Everywhere event, the company has revealed the forthcoming release of 5th Gen Xeon processors, featuring AI acceleration in every core and expected to hit the market in 2024. Unveiled by Intel CEO Pat Gelsinger, these processors, previously codenamed Emerald Rapids, mark a significant advancement in computing. 

Designed to cater to AI, high-performance computing, networking, storage, databases, and security needs, they aim to enhance performance while minimising the total cost of ownership (TCO).

Intel Xeon powered data centres would be used by Microsoft, Google Cloud, and IBM, and many more would integrate them.

Sandra Rivera, Intel’s executive vice president and general manager of Data Center and AI Group, underscored the importance of this development. “Built for AI, our 5th Gen Intel Xeon processors provide greater performance to customers deploying AI capabilities across cloud, network, and edge use cases. We’re launching 5th Gen Intel Xeon on a proven foundation that will enable rapid adoption and scale at lower TCO,” she stated.

Key features of the 5th Gen Xeon processors include AI acceleration in every core, optimizing them for AI workloads and delivering up to 42% higher inference performance with minimal latency on large language models. This enables end-to-end AI processing without the need for additional accelerators, making AI tasks more efficient.

In terms of general compute performance, the processors offer an average gain of 21% compared to the previous generation. They also boast a 36% increase in performance per watt across various customer workloads, translating to significant efficiency improvements.

These processors support up to 64 cores per CPU, enhanced last-level cache, eight channels of DDR5, and higher memory transfer speeds. These improvements contribute to overall enhanced performance and bandwidth. Additionally, they ensure compatibility with CXL Type 3 workflows through leading cloud service providers.

Furthermore, the processors come equipped with security enhancements, featuring Intel Trust Domain Extensions (Intel TDX). This provides increased confidentiality and security at the VM level, ensuring enhanced privacy and control over data.

Looking ahead, the 5th Gen Xeon processors, which are pin-compatible with the previous generation, are set to be available in single- and dual-socket systems from leading OEMs such as Cisco, Dell, HPE, Lenovo, and others by the first quarter of 2024. Major cloud service providers will announce the availability of instances based on these processors throughout the year.

Intel remains committed to its roadmap, with plans to introduce Sierra Forest, emphasising E-core efficiency with up to 288 cores, in the first half of 2024. Following closely will be Granite Rapids, focusing on P-core performance.

The company has also announced the launch of AI PCs, to build and run AI on every PC. Gelsinger announced the company’s partnership with Dell, HP, Lenovo, Supermicro, and Microsoft for onboarding the chips on their devices.

The post Intel Releases 5th Gen Xeon Processors appeared first on AIM.

Just as expected, Intel has announced the launch of AI PCs, to build and run AI on every PC. Pat Gelsinger, CEO of Intel, at the AI Everywhere event announced the company’s partnership with Dell, HP, Lenovo, Supermicro, and Microsoft for onboarding the chips on their devices.

This is made possible by the Intel Core Ultra chips announced at the event along with the Intel Arc GPUs. Featuring a maximum of 8 XE GPU cores, the performance is akin to having an integrated Intel Arc 370M discrete GPU directly within the processor, albeit without the presence of dedicated video memory.

The announcement, first made at Intel Innovations a couple of months ago, highlights the processors as the company’s pioneering venture into incorporating a neural processing unit (NPU) into their chips.

Gelsinger added that Intel is dedicated to developing technologies that enable seamless integration and efficient operation of AI across various applications, both in the cloud and increasingly on local devices such as PCs and edge devices.

“We think 2024 marks the era of AI PC and that will be the star of the show in this coming year,” said Gelsinger. 

The AI PC concept aims to bring AI capabilities directly to personal computers, allowing users to run generative AI chatbots, like ChatGPT, locally, without relying on cloud data centres, and even for inference models such as Llama 2. This development is particularly significant for enhancing data security, a crucial aspect for business and enterprise users.

Intel aims to deliver 100 million client processors, which is five times more than any of the competitors. 

Furthermore, the Core Ultra processors bring about a paradigm shift with three major architectural features introduced to Intel’s laptop portfolio. Firstly, there is a departure from the conventional single-silicon-die design to a multi-chiplet-module (MCM) design. This novel approach involves bonding several smaller silicon chiplets together to create a single chip, offering increased versatility in chip functionality compared to previous processor generations.

AMD also announced at its Advancing AI event that it’s AI PC idea by partnering with Microsoft, Meta, and several OEM companies. At the AMD Advancing AI event, the company has unveiled various lines of AI announcements including the Instinct MI300X AI accelerators to compete with NVIDIA H100, updates to ROCm, and Ryzen AI PCs for on-device computing.

The post Intel Releases Core Ultra, Arc GPUs for Bringing AI to Every PC appeared first on AIM.

Intel is all set for its ‘AI Everywhere’ event coming on December 14, with a bunch of AI announcements, which it touts would usher in its top place in the generative AI realm.

The Chip company is planning the release of its Gaudi3 AI accelerator chip at its event, which would be a major game changer for the company. CEO Pat Gelsinger believes that the supercomputer that Intel is building will be the largest in Europe. He even hinted at Gaudi3, which according to him, would be two times faster than Gaudi2. 

To put it in perspective, this comes after Intel has already been providing Gaudi2 AI chips for training models. Interestingly, Gaudi2 works 2.4 times faster than the NVIDIA A100, and is almost coming close to the H100 Hopper GPU. 

Gaudi3 is expected to arrive with a 5nm chip. The accelerators are set to provide a significant boost with up to 4 times the BFloat16 capabilities, double the compute power, 1.5 times the network bandwidth, and a 1.5 times increase in HBM capacities (144 GB compared to 96 GB). Looking ahead to 2025, the successor to Gaudi3, Falcon Shores, will merge the AI capabilities of Gaudi with the powerful GPUs from Intel, all within a single package.

All about AI?

Intel is embarking on a bold journey in AI to compete with NVIDIA, AMD. At Intel Innovation 2023 in September, the company revealed its ambitious roadmap for the next few years, making it clear that they are going all in on AI. This includes their monster 288-core Xeon CPU, based on Emerald Rapids architecture, that’s coming next year.

The company has also announced that Stability.AI would purchase a Gaudi2-based AI supercomputer with Xeon processors overseeing 4,000 Gaudi2 accelerators. 

What caught the industry’s attention were the processors in the pipeline such as Arrow Lake, Lunar Lake, and Panther Lake scheduled for 2024 and 2025. These processors represent a significant leap forward in Intel’s pursuit of technological excellence.

Intel is planning to onboard another version of AI accelerator superchip, Falcon Shores 2, by 2026. “We have a simplified roadmap as we bring together our GPU and our accelerators into a single offering,” CEO Pat Gelsinger said. Though this is a far out vision, the close by announcements are no joke as well.

Furthermore, Intel’s Falcon Shores chips were originally conceived as a fusion of CPU and GPU cores, representing the company’s inaugural venture into the ‘XPU’ architecture for high-performance computing. Nonetheless, a few months ago, Intel astounded the industry by opting for a GPU-only approach and deferring the chip’s release until 2025. The company’s voyage into the realm of AI and GPUs has encountered a series of twists and turns.

The focus on developers

The announcement on Meteor Lake, set to launch on December 14, was undoubtedly a headline-grabber because as CEO Pat Gelsinger said, the processor will “power-efficient AI acceleration and local inference on the PC”.

Gelsinger emphasised the company’s commitment to engineering excellence and showcased its efforts to democratise AI with the “AI PC” concept, which is similar to AMD’s Ryzen AI PCs. This innovation is made possible by Intel’s forthcoming “Meteor Lake” laptop chip, which incorporates new AI data-crunching features. 

The AI PC concept aims to bring AI capabilities directly to personal computers, allowing users to run generative AI chatbots, like ChatGPT, locally, without relying on cloud data centres, and even for inference models such as Llama 2.

Moreover, to compete with AMD’s ROCm and NVIDIA’s CUDA, developers can utilise the oneAPI programming model to build and optimise AI and high-performance computing workloads. In addition, Intel has revealed Project Strata, a commercial software platform set to launch in 2024, aimed at supporting distributed edge infrastructure and applications with modular building blocks and premium services.

Focusing on developers, Intel had already announced the general availability of its Intel Developer Cloud platform, offering developers the opportunity to test and deploy AI and high-performance computing applications with the latest CPUs, GPUs, and AI accelerators. 

The platform includes access to fifth-generation Xeon Scalable processors, Intel Data Center GPU Max Series, Intel Gaudi2 deep learning processors, and Intel software and tools. 

All of this clearly shows that Intel might make a significant mark on December 14, as the road it’s taking with Gaudi3, upgrades to Xeon for AI PC, and the prowess of Gaudi2, all hints towards just that.

While Intel navigates its strategic adjustments, it’s noteworthy that NVIDIA has also taken a substantial leap by venturing into the CPU market with the GH200 supercomputer, which it has already started rolling out. This expansion into CPUs complements NVIDIA’s existing prowess in GPUs and AI technologies, while venturing into the CPU market.

AMD is also all-in on AI compute by partnering with Microsoft, Meta, and several OEM companies. At the AMD Advancing AI event, the company has unveiled various lines of AI announcements including the Instinct MI300X AI accelerators to compete with NVIDIA H100, updates to ROCm, and Ryzen AI PCs for on-device computing. 

The post Can Intel Play Catch-Up with NVIDIA and AMD? appeared first on AIM.

NVIDIA has been shining all along with the latest Q3 earnings reflecting the unstoppable growth of the tech giant. The latest earnings reported a revenue of $18.12 billion which was a 206% increase YoY and 34% from the previous quarter. The company even attributed the phenomenal growth in revenue to its continued ramp of NVIDIA HGX platform along with end-to-end networking via InfiniBand. 

NVIDIA has called out the contribution of networking that has now exceeded $10 billion annualised revenue run rate, nearly tripling from the previous year. This is attributed to the rising demand for InfiniBand which witnessed a fivefold increase YoY.

A Complete Architecture

InfiniBand, which is considered critical for gaining the scale and performance needed for training LLMs, when combined with NVIDIA HGX forms the foundational architecture for AI supercomputers and data centre infrastructures. InfiniBand is commonly used in supercomputing environments for interconnecting servers. The biggest advantage is its ability to provide low latency and high-bandwidth communication that is crucial for parallel processing tasks. With extreme-size datasets and ultra-fast processing of high-resolution simulations, NVIDIA’s Quantum InfiniBand Switches are said to match these needs with lower cost and complexity. 

A few months ago, NVIDIA had reached breakthrough performance with their leading H100 chip. The tests were run on 3,584 H100 GPUs that were connected with InfiniBand as they allowed GPUs to deliver performance at standalone and scale levels. Thereby, proving its prowess when combined with high performing networking capabilities. 

InfiniBands : The Preferred Choice

Speaking about the future of InfiniBands, Jensen Huang said that the vast majority of the dedicated large scale AI factories standardise on InfiniBand, and it’s not only because of data rate and latency but “the way traffic moves around the network” is important. He also called it a ‘computing fabric.” 

Comparing it to Ethernet, Huang talks about the huge difference between the two. With NVIDIA investing $2 billion in infrastructure for AI factories, any form of variance, such as 20 or 30% in overall effectiveness will result in millions of dollars of change in value which accumulate as significant costs over the next 4-5 years. 

Huang calls InfiniBand’s value proposition ‘undeniable for AI factories.’ However, Ethernet is not ruled out. While Infinibands are used for cases that require high bandwidths with low latency, ethernet finds applicability in other scenarios. 

Ethernet, a widely used general-purpose networking technology for wired local area networks (LAN), is suitable for a broad range of applications, more geared towards connecting terminal devices. However, its capabilities cannot be matched with InfiniBands. 

Interestingly, NVIDIA also offers gateway appliances connecting InfiniBand data centres to Ethernet-based infrastructures and storage. NVIDIA will also release Spectrum-X in Q1 next year, an Ethernet offering that is said to achieve 1.6x higher networking performance when compared to other available Ethernet technologies. 

In terms of functionality, Intel’s Omni Path Architecture (OPA) was designed for high-speed data transfer and low latency communication in HPC environments. It was released in 2016, however, it was discontinued in 2019. Cisco on the other hand, has ethernet-based switches but nothing in the HPC space. 

An Integrated Expansion

With GPU and networking offerings, enterprises are now given the choice of integrating their whole architectural framework from NVIDIA products. In addition to speaking about NVIDIA’s partnerships with Reliance, Infosys and Tata, the company mentioned their collaborations with organisations for optimising InfiniBands in their AI compute needs.  

In the earnings call, NVIDIA spoke about its partnership with Scaleway, a French private cloud provider that will build their regional AI cloud based on NVIDIA H100 InfiniBand and AI Enterprise Software to power AI advancements across Europe. 

Furthermore, Julich, a German supercomputing centre, also announced its plans to build their next-gen AI supercomputer using close to 24,000 Grace Hopper Superchips and Quantum-2 InfiniBand, elevating it to world’s most powerful AI supercomputer with over 90 exaflops of AI performance. 

Interestingly, Microsoft Azure uses over 29,000 miles of InfiniBand cabling. Infiniband enabled HB and N-series’ virtual machines are utilised by Microsoft for achieving HPC with cost efficiency. 

Bundling networking and GPU, NVIDIA is boosting its growth and stance in the supercomputer market. Going by the lack of alternatives to NVIDIA Infinibands, it looks like the company’s dominance is going to be further enhanced, ultimately making it indispensable for companies looking to utilise GPU and networking. 

The post NVIDIA Rides High on InfiniBands appeared first on AIM.

Intel joins the Make in India brigade. The company has unveiled a strategic partnership with eight prominent Electronics Manufacturing Services (EMS) companies and Original Design Manufacturers (ODMs) to bolster laptop manufacturing in India.

The move aims to harness Intel’s extensive industry knowledge to lay the groundwork for a robust laptop manufacturing sector in the country, in alignment with the Make in India initiative.

The collaborative effort with Intel involves firms like Bhagwati Products Ltd, Dixon Technologies India Ltd, Kaynes Technology India Ltd, Optiemus Electronics Ltd, Panache Digilife Ltd, Smile Electronics Ltd, Syrma SGS Technology Ltd, and VVDN Technologies Private Ltd.

For some of these companies, this venture signifies their inaugural foray into laptop manufacturing, reflecting Intel’s commitment to empower the Indian manufacturing ecosystem to cater to both domestic and global demand.

As part of this collaboration, Intel will leverage its expertise to facilitate the production of complete entry-level laptops in India, employing state-of-the-art Surface Mount Technology (SMT) assembly lines, implementing quality control processes for components, and benchmarking finished products. Intel also offered support to ODMs across both Semi Knocked Down (SKD) and Completely Knocked Down (CKD) manufacturing processes.

“It is our Prime Minister’s goal that the Indian Electronics Ecosystem should have deep and broad capabilities, and that Indian Electronics Manufacturing Companies should grow, scale, and expand their footprint as trusted players in the Electronics Global Value Chains,” stated Rajeev Chandrasekhar, Minister of State for electronics and IT, skill development, and entrepreneurship.

“By enabling the laptop manufacturing process – from surface mount technology assembly to finished product – we are not only meeting the demands of the Make in India initiative but also contributing to the technological progress of the nation,” remarked Santhosh Viswanathan, VP & MD, India region, Intel.

Intel is set to host the India Tech Ecosystem Summit in November, which will bring together numerous local manufacturers to showcase a broader range of devices manufactured in India. 

The post Intel Collaborates with Indian Manufacturers to Make Laptops in India appeared first on AIM.

Intel has done better than expected in the third quarter, with an increase in revenue and margins from $12.9 billion in the last quarter to $14.2 billion, driven by a stabilising server chip business and stronger PC market conditions. 

The Q3 revenue was down by 8% YoY, indicating a massive recovery from the last quarter which was down by 15% YoY. Gross margins improved, and they’ve secured new customers for chip manufacturing, with more deals on the horizon. 

The global PC market is on the rebound, and the company’s forecast for Q4 includes adjusted revenue expectations exceeding Wall Street estimates for both revenue and profit per share, from $14.6 billion to $15.6 billion.

Despite recent challenges, such as heavy manufacturing investments impacting gross margins, Intel is making progress with its turnaround plans.

For the upcoming quarters, Intel is betting big on its foundry services, the company’s nascent chip-manufacturing business, which mustered a meteoric $311 million in revenue, growing 299% from the previous year.

A Slew of Partnerships 

The tech giant announced that a major customer committing to Intel 18A and Intel 3 with a prepayment allowed the company to accelerate its plans to build two new leading-edge chip factories at its Ocotillo campus in Chandler, Arizona. 

The customer could very well be American electronic design automation company Synopsys, which inked a deal with Intel in August of 2023.

Additionally, IFS and Tower Semiconductor got into an agreement wherein Intel would provide foundry services and 300 mm manufacturing capacity to help Tower serve its customers globally, using Intel’s advanced manufacturing facility in New Mexico. The move came after the merger plans of the two entities failed.

Ambitions and Challenges

“Our ambition is to be the No. 2 foundry in the world by the end of the decade,” Randhir Thakur, ex- president of Intel Foundry Services had reiterated in an interview. CEO Pat Gelsinger’s move to open Intel’s fabrication facility to external clients was an important step in that direction, representing a strategic shift.

To attain this goal, Intel has embarked on a significant investment plan, including allocating $20 billion to build two new facilities in Arizona, with an additional $20 billion earmarked for a site in Ohio. 

However, the objective isn’t solely propped on revenue ranking, as there are several key elements to consider.

Firstly, to achieve the status of the second-largest foundry, Intel aims to become a technology leader, with rivals TSMC and Samsung standing tall in the way. These two industry giants have made considerable progress in both R&D and capacity expansion over the past decade, outpacing Intel. 

In the fast-paced semiconductor industry, companies must advance to the next process node every two years or faster to stay relevant. Intel, unfortunately, lagged in this aspect and must work to regain its competitive edge.

Advancing in semiconductor technology is just the first step. The ability to turn cutting-edge chip technology into high-volume production with a low rate of defects, known as yield, is equally vital. 

The semiconductor industry is highly competitive, with TSMC and Samsung consistently advancing to the next node and achieving rapid revenue growth. In contrast, companies like United Microelectronics Corp (UMC) struggle to keep pace.

This divide has resulted in a market bifurcation where TSMC and Samsung dominate the high-end chip production, which includes chips used in smartphones, AI servers, and crypto miners. 

Meanwhile, other companies handle low-end production, including chips for automotive applications, smart speakers, and industrial robots. Approximately 50% of the foundry market comprises products manufactured at 16nm and smaller process nodes, almost exclusively produced by TSMC and Samsung.

Intel’s challenge is not only to advance technologically but also to attract leading clients like Apple, NVIDIA, and AMD. Intel has to prove that it can be trusted with both chip designs and large-scale, on-time, and low-defect chip production. 

Apple’s decision to switch from Intel as a Mac processor supplier in favour of in-house processors underscores the challenges of retaining leading clients.

If Intel doesn’t manage to keep up with rivals, it may drop back by several development cycles within half a decade. This could result in a three-tier market: Intel is in the middle, not charging as much as the leaders but lacking the low-cost advantages of smaller players. 

Intel might have a profitable niche if it can convince strategic clients and the US government that American-made chips are vital for security-sensitive industries like defence, aerospace, and data management.

Ultimately, Intel’s success in this endeavour depends on both technological breakthroughs and effective salesmanship. It may need to rely on government support and convince customers that being almost leading in technology is sufficient for their needs. 

Intel’s journey to become the world’s second-largest foundry involves multiple challenges, and its position within the market is likely to be determined once it can prove itself to clients and stakeholders.

Nonetheless, a multitude of partnerships and a near-300% profit seems more than encouraging and is a nod to the decision to move in this direction. After all, Intel might have found an avenue to scale.

The post Why Intel is Betting Big on Chip Manufacturing appeared first on AIM.

Across the spectrum, from cloud service providers to AI labs and startups, there’s a fervent desire for access to the cutting-edge Graphics Processing Units (GPUs) available. However, the market is currently grappling with a scarcity of these high-end GPUs, which are primarily dominated by a single company, NVIDIA. This heightened demand from enterprises has led to a scarcity, resulting in soaring prices.

This is a challenge for the industry with many opining that the shortage could even stifle AI innovation.  Hence, what the industry needs is competition. While NVIDIA, being a pioneer in this field, remains the leader, other GPU vendors such as Intel and AMD are making great strides and closing the gap between NVIDIA. However, choosing between multiple GPU vendors remains a complex task. 

Need for an open software architecture

If your software or applications are optimised for one vendor’s GPUs, it may be challenging to transition to GPUs from one vendor to another without significant code modifications and testing. Additionally, GPU drivers and application programming interfaces (APIs) are vendor-specific. Applications that use these APIs, like CUDA or OpenCL, may not be compatible with GPUs from other vendors without significant modification. This can result in a lock-in period where switching GPUs becomes complex and costly.

Moreover, GPU vendors often provide software development kits (SDKs), libraries, and tools tailored to their GPUs. Developers may rely on these vendor-specific software components for tasks like GPU programming (e.g.CUDA for NVIDIA GPUs). Hence, switching to GPUs from a different vendor may require rewriting or adapting software to work with their software stack.

Hence, an open software architecture that facilitates the selection of GPU providers would undeniably be a significant boon to the AI community, Mohammed Imran K R, chief technology officer at E2E Networks, told AIM. Besides making it easier to choose between GPU vendors, an open software architecture would eliminate the constraints of long-term vendor lock-in, allowing AI researchers and developers to choose GPUs based on their specific requirements. 

“It would also lead to a more competitive environment, pushing GPU manufacturers to innovate and offer better hardware options for AI workloads. It would also drive cost efficiency, as organisations could select GPUs based on both performance and cost, thus optimising their resources,” he said.

Furthermore, an open infrastructure would encourage collaboration within the AI community. Standardised tools and interfaces would make it easier for developers and researchers to work with different GPU platforms, potentially accelerating advancements in AI technology. “Additionally, this approach aligns with industry trends favouring open-source solutions and interoperability, empowering companies to construct adaptable technology stacks,” Shivam Arora, marketing manager at Compunnel told AIM.

Nonetheless, it is also essential to consider that developing and maintaining such an open infrastructure would require coordination from the GPU vendors, software developers and AI community. “While flexibility will be derived, performance optimisation could be an issue,” Sanjay Lodha, Chairman & Managing Director of Netweb Technologies told AIM.

OpenCL, ROCm and oneAPI

One could argue that OpenCL is one such open software architecture that already exists. Launched in 2009 by Apple and the Khronos Group to offer a standard for heterogeneous computing, OpenCL might be a viable option, but it does come with its own set of challenges. OpenCL allows you to write programs that can be executed on various GPU architectures from different vendors. “Even though OpenCL is gaining traction, it is still limited and may not provide the same level of optimisation as a vendor-specific tool like CUDA from NVIDIA,” Lodha said.

From an AI technology development standpoint, OpenCL currently has several drawbacks when compared to CUDA, with one critical aspect being that the majority of the latest research, models, and frameworks assume CUDA as the default GPU programming platform. “Additionally, achieving true cross-vendor portability can be challenging with OpenCL, as different GPU manufacturers implement it with varying degrees of compliance and performance,” Imran said.

In fact, a study comparing CUDA programs with OpenCL on NVIDIA GPUs showed that CUDA was 30% faster than OpenCL. Simultaneously, AMD’s ROCm, which is also an alternative to CUDA, is also making great strides. Interestingly, CUDA code can be converted to ROCm code using the HIP (Heterogeneous-Computing Interface for Portability) tools provided by AMD. Another interesting development in the last few years is oneAPI. While ROCm targets both AMD and NVIDIA GPUs, oneAPI applications can run on GPUs from Intel, NVIDIA and AMD, making both a viable option. 

Moving away from CUDA

While enterprises might look into alternate vendors like Intel, AMD or even China-based Huawei, but for the industry to move away from CUDA could be challenging. “It involves rewriting or adapting existing code, potentially causing disruptions and requiring significant retraining of developers. However, the industry’s increasing interest in open-source alternatives indicates a growing willingness to embrace change. The difficulty of this transition ultimately depends on the specific needs and objectives of the company and its commitment to open-source principles,” Arora said.

Lodha on the other hand is a bit more sceptical. He believes it will be immensely difficult for the AI community to move away from CUDA towards a more open software architecture because many machine learning models have been trained using CUDA code. “This means that researchers and developers would need to rewrite their code in order to use a different GPU programming framework.”

Nonetheless, he believes that the benefits of moving to a more open-source GPU programming framework outweigh the costs. He further stated that an open-source framework would make it easier for researchers and developers to compare the performance of different GPUs and to choose the GPU that is best suited for their needs. It would also make it easier for vendors to compete with each other, which would lead to lower prices and better products.

“I think the best way to move from CUDA to a more open-source alternative is to transition gradually. Researchers and developers could start by writing new code in an open-source framework, such as OpenCL or RoCm. They could also start porting existing CUDA code to an open-source framework. There are already tools that are being used but still programming efforts are required.”

Imran also concurs. He thinks ensuring compatibility with other components of the software stack and achieving true cross-vendor portability is challenging at the moment. “However, in the long run, we believe that there will be alternatives and there are compelling reasons for it, including reducing vendor lock-in, promoting interoperability, and contributing to a more diverse GPU ecosystem.”

The post The Urgent Need for an Open GPU Architecture appeared first on AIM.

Intel is embarking on a bold journey in AI to compete with NVIDIA, AMD, and Apple. At Intel Innovation 2023, the company revealed its ambitious roadmap for the next few years, making it clear that they are going all in on AI. This includes their monster 288-core Xeon CPU, based on Emerald Rapids architecture, that’s coming next year.

The announcement on Meteor Lake, set to launch on December 14, was undoubtedly a headline-grabber because as CEO Pat Gelsinger said, the processor will “power-efficient AI acceleration and local inference on the PC”.

Gelsinger emphasised the company’s commitment to engineering excellence and showcased its efforts to democratise AI with the “AI PC” concept. This innovation is made possible by Intel’s forthcoming “Meteor Lake” laptop chip, which incorporates new AI data-crunching features. The AI PC concept aims to bring AI capabilities directly to personal computers, allowing users to run generative AI chatbots, like ChatGPT, locally, without relying on cloud data centres. 

To showcase the capabilities of the AI PC, Gelsinger, along with RewindAI CEO Dan Siroker, demoed the capabilities of running Rewind on Windows, which was powered by Llama 2 and running entirely on the device. This will also be available on Intel-based MacBooks.

Focusing on developers, Intel has announced the general availability of its Intel Developer Cloud platform, offering developers the opportunity to test and deploy AI and high-performance computing applications with the latest CPUs, GPUs, and AI accelerators. The platform includes access to fifth-generation Xeon Scalable processors, Intel Data Center GPU Max Series, Intel Gaudi2 deep learning processors, and Intel software and tools. 

Moreover, developers can utilise the oneAPI programming model to build and optimise AI and high-performance computing workloads. In addition, Intel has revealed Project Strata, a commercial software platform set to launch in 2024, aimed at supporting distributed edge infrastructure and applications with modular building blocks and premium services.

A Three-Pronged Performance Approach

What caught the industry’s attention were the processors in the pipeline such as Arrow Lake, Lunar Lake, and Panther Lake scheduled for 2024 and 2025. These processors represent a significant leap forward in Intel’s pursuit of technological excellence.

Intel is employing a three-pronged approach to benchmark their processors against competitors like Apple and even NVIDIA. This approach includes testing CPUs, GPUs, and NPUs within a fixed power budget. 

One of Intel’s key strategies for regaining leadership is its focus on chip manufacturing. Gelsinger announced that Intel’s fabrication facilities would begin producing Panther Lake processors in early 2024. This move comes as part of Intel’s ambitious plan to accelerate its chipmaking progress. The company is investing heavily in advanced manufacturing processes, including extreme ultraviolet light (EUV), to etch finer features onto silicon wafers. This technology allows for the production of smaller, more efficient processors.

While it seems like NVIDIA has dominated this space, Intel is making significant strides in the right direction. The company has also announced that Stability.AI would purchase a Gaudi2-based AI supercomputer with Xeon processors overseeing 4,000 Gaudi2 accelerators. 

To contextualise Intel’s focus on AI, we must first acknowledge its competition, particularly with Apple. For years, Intel supplied processors to Apple’s Mac lineup. However, with the introduction of Apple’s M1 and M2 chips, the tech giant decided to part ways with Intel. Apple’s in-house processors have received widespread acclaim for their performance and power efficiency.

However, Intel, not one to back down from a challenge, has made it clear that improving both performance and energy efficiency is a top priority. While specific details about Meteor Lake’s performance remain undisclosed, Intel promises significant advancements in processing, graphics, and AI capabilities. Gelsinger emphasised that these improvements would put Intel’s offerings on par with the best that Apple and other competitors have to offer.

Making big bets

Gelsinger believes that the supercomputer that Intel is building will be the largest in Europe. He even hinted at Gaudi3, which according to him, would be two times faster than Gaudi2. It is expected to be released by the end of 2025. This comes after Intel has already been providing Gaudi2 AI chips for training models. Interestingly, Gaudi2 works 2.4 times faster than the NVIDIA A100, and is almost close to the H100 Hopper GPU. 

Intel’s plan to converge its data centre GPU and Gaudi road maps with the Falcon Shores GPU, set to release around the same time, looks like a robust plan. Falcon Shores chips were originally conceived as a fusion of CPU and GPU cores, representing the company’s inaugural venture into the ‘XPU’ architecture for high-performance computing. 

“Simply put, our roadmap is extremely robust, and we are executing aggressively to bring this together,” Gelsinger said.

The post Intel Goes All in on AI appeared first on AIM.

NVIDIA’s AI hardware remains a highly coveted product in the Silicon Valley. In fact, the Jensen Huang-led company currently enjoys a near-monopoly in the GPU market. Nevertheless, Intel, a company with a long history of chip dominance, is progressively narrowing the gap with NVIDIA. This competition between the two giants could prove beneficial for the market as a whole.

In the recently released ML Perf benchmark test, NVIDIA dominated the charts and won every benchmark, however, surprisingly, Intel finished a close second. The company provided performance results for the Habana Gaudi2 accelerators, 4th Gen Intel Xeon Scalable processors, and Intel Xeon CPU Max Series. 

Intel beats NVIDIA in Vision Models 

Since its acquisition of AI chipmaker Habana Labs in 2019 for USD 2 billion, Intel has tried to break into the AI compute market and is making significant strides. Interestingly, Gaudi2’s performance surpassed that of NVIDIA’s H100 on a state-of-the-art vision language model on Hugging Face’s performance benchmarks. “Optimum Habana v1.7 on Habana Gaudi2 achieves 2.5x speedups compared to A100 and 1.4x compared to H100 when fine-tuning BridgeTower, a state-of-the-art vision-language model,” a Hugging Face blog post states. 

This performance improvement, according to Intel, relies on hardware-accelerated data loading to make the most of your devices. Furthermore, Intel believes these recent results serve as a strong confirmation that presently, Intel stands as the primary and most compelling alternative to NVIDIA’s H100 and A100 for AI computing requirements. 

Intel’s claim is based on Gaudi2’s strong performance on the ML Perf benchmarks as well. When it comes to LLMs, Gaudi2 delivers compelling performance against NVIDIA’s H100, with H100 showing a slight advantage of 1.09x (server) and 1.28x (offline) performance relative to Gaudi2. Given NVIDIA’s first-mover advantage and Intel’s track record, it’s a significant development. 

Why it matters

What Intel would bring with its Gaudi2 processors is competition to the market, besides catering to the GPU shortage. Today, enterprises want to get their hands on the NVIDIA H100 GPUs, because it is the fastest AI accelerator on the market. Moreover, NVIDIA has established its dominance in the market through forward-thinking strategies, a meticulously planned and well-documented software ecosystem, and sheer processing prowess. However, what Intel seeks to challenge is the prevailing industry narrative that generative AI and LLMs can solely operate on NVIDIA GPUs.

Currently, NVIDIA’s GPUs come at an exorbitant price. A single H100 could cost around USD 40,000. While Gaudi 2 is not only closing the gap with NVIDIA, but also claims to be cheaper than NVIDIA’s processors. This is a welcome news even though Intel has not specifically revealed its price point. “Gaudi2 also provides substantially competitive cost advantages to customers, both in server and system costs,” Intel said in a blog post. 

Gaudi2 is powered by a 7nm TSMC processor, compared to NVIDIA’s 5nm Hopper GPU. However, the next generation of Gaudi could be powered by a 5nm chip and likely be released by year end. Moreover, according to Intel, the integration with FP8 precision quantisation would make Gaudi2 even faster for AI interference. 

Tarun Dua, CEO at E2E Networks, also told AIM that Intel’s entry into the GPU space is great news because competition is always good for the market. Other players entering the market with relatively cheaper alternatives would make GPUs more accessible. AMD and China-based Huawei are also working to introduce GPUs to the market to meet the growing demand. Moreover, constrained supply chains could hamper innovation.

“However, for Intel to truly compete with NVIDIA, it will need to build an entire plug-and-play ecosystem that NVIDIA has built over the years. It may take some time before we witness Intel operating at production scales comparable to NVIDIA, particularly in the training aspect. However, in terms of interference, we might observe their entry sooner,” Dua said.

Challenges galore 

While Intel is closing the gap, NVIDIA, on the other hand, continues to leap higher. The GPU maker has unveiled an updated TensorRT software designed specifically for LLMs. This software promises to deliver significant enhancements in both performance and efficiency during inference processing applicable to all NVIDIA GPUs. The software upgrade could supercharge NVIDIA’s most advanced GPUs, which could further widen the gap between NVIDIA and Intel. 

Besides the pure performance of its GPUs, NVIDIA moat is its software stack. NVIDIA released its first GPU in 1999 and in 2006 NVIDIA developed CUDA, touted as the world’s first solution for general computing on GPUs. Since then, the CUDA ecosystem has grown drastically. 

“This dominance extends beyond just hardware, as it heavily influences the software, frameworks and ecosystem that surrounds it. The prevalence of NVIDIA GPUs in all the recent AI / ML advances has led to the centralisation of research, models, and software development around CUDA. Nearly every AI researcher currently chooses CUDA as a default due to this,” Mohammed Imran K R, CTO at E2E Networks, told AIM.

To compete with CUDA, Intel is in the process of shifting its developer tools to LLVM to support cross-architecture compatibility. Additionally, they are adopting a specification known as oneAPI for enhanced accelerated computing. This move aims to lessen NVIDIA’s control by reducing its reliance on CUDA.

“For it to become pervasive, accelerated computing needs to be standards-based, scaleable, multi-vendor and ideally multi-architecture. We set out four years ago to do that with oneAPI… and we’ve got to the point where we’re becoming productive for developers,” Joe Curley, VP and general manager for software products at Intel said in an interview. Moreover, an open-source infrastructure that makes it easy to choose among GPU providers is also the need of the hour.

The post Why Intel Closing the Gap with NVIDIA is Good News appeared first on AIM.

Everyone knows why NVIDIA is on the top of the market in generative AI when there are competitors like Intel and AMD who are also making strides. Now the behemoth Intel is going all in into the AI hardware segment, and it might have just cracked it.

Intel is planning to onboard another version of AI accelerator superchip, Falcon Shores 2, by 2026. “We have a simplified roadmap as we bring together our GPU and our accelerators into a single offering,” CEO Pat Gelsinger said. 

The recently released Intel Xeon Max 9480 combines 56 cores and is not a standard DDR5 memory, but a 64 GB HBM2e, which is on par with the ones used in GPUs and AI accelerators. Interestingly, Intel believes that these GPUs would be mostly used for inference based tasks, and not actually training AI models. 

More leaks about the upcoming 14th Gen Meteor Lake processor also suggest that the CPU might have a DDR5 memory, which is also similar to Apple’s M2 chip design. Moreover, it is also expected that AI will play a major role in the Meteor Lake CPUs. Much is awaited at the upcoming Intel Innovation 2023 event on September 19.

All roads lead to AI

While Intel navigates its strategic adjustments, it’s noteworthy that NVIDIA has also taken a substantial leap by venturing into the CPU market with the GH200 supercomputer. This expansion into CPUs complements NVIDIA’s existing prowess in GPUs and AI technologies, while venturing into the CPU market.

Furthermore, Intel’s Falcon Shores chips were originally conceived as a fusion of CPU and GPU cores, representing the company’s inaugural venture into the ‘XPU’ architecture for high-performance computing. Nonetheless, a few months ago, Intel astounded the industry by opting for a GPU-only approach and deferring the chip’s release until 2025. The company’s voyage into the realm of AI and GPUs has encountered a series of twists and turns.

This comes after Intel has already been providing Gaudi2 AI chips for training models. Interestingly, Gaudi2 works 2.4 times faster than the NVIDIA A100, and is almost coming close to the H100 Hopper GPU. 

On the flip side, Intel’s decision to decelerate its GPU release cadence could potentially place it at a disadvantage against more advanced architectures like NVIDIA Grace Superchips and AMD’s Instinct MI300, both slated for launch in 2023. This strategic choice may hinder Intel’s competitiveness in the HPC market.

Intel is convinced with two AI markets

One that deals with the infrastructure, for which the company has the Habana Labs Gaudi. The other is for inference, which according to Intel can be adequately done on a CPU like Xeon. 

This seemed like an almost good approach until NVIDIA jumped onto the same wagon. At NVIDIA’s recent financial call, Jensen Huang said that the company plans to introduce L40S, a GPU that is specifically designed for fine-tuning and inference. Given that people are already using NVIDIA H100s for training, shifting to Intel’s Xeon processors might be a big leap to make. 

Amid these strategic shifts, in May Intel had announced a strategic collaboration with the Boston Consulting Group (BCG) to facilitate generative AI. This partnership aimed to harness Intel’s AI hardware and software to craft tailor-made generative AI solutions for enterprises, all while ensuring the sanctity of data privacy and security.

Cut to August, Anthropic announced its partnership with BCG for bringing responsible generative AI for enterprise clients. But after that, NVIDIA and Microsoft also made an investment in Anthropic, which makes all of this a little confusing. This proves that AI is for everyone to take. This might be a hint that the Google-backed startup might be leveraging Intel supercomputers for building generative AI, which is quite rarely heard given the NVIDIA GPU dominance. 

Intel GPUs, NVIDIA CPUs

You read that right. All of these strategic shifts were interpreted as Intel’s departure from direct competition with AMD’s Instinct MI300 and NVIDIA’s Grace Hopper processors, both of which boast a combined CPU+GPU design. NVIDIA went into the CPU business in March and it left people wondering, what else does the GPU giant want to take on Intel with.

On the other hand, Intel has shed light on the reasoning behind this strategic reconfiguration. While the initial plan for Falcon Shores permitted flexible CPU/GPU configurations, Intel emphasised the significance of enabling customers to utilise various CPUs, including those from rivals like AMD and NVIDIA. But given the announcements around Gaudi2, a potential Gaudi3, and NVIDIA venturing into CPUs, Intel might be able to take a bigger slice of the GPU market soon.

The rivalry between Intel and NVIDIA, encompassing both CPUs and GPUs, is poised to intensify, potentially reshaping the landscape of AI and HPC. Intel is the best CPU to buy, and NVIDIA is the best GPU to buy. This is widely believed. But it might take a turn soon given that the conversation about computing has almost shifted around AI.

The post Intel Soon to be on Par with NVIDIA appeared first on AIM.

By now, it is clear that no matter who wins the AI race, the biggest profiteer is NVIDIA. It’s common knowledge that the company is a market leader in the hardware category with its GPUs being widely used by all AI-focused companies in the world. That’s not all. NVIDIA, the biggest chip company in the world, is leading the battle from the software side of things as well, with its CUDA (Computing Unified Device Architecture) software. 

CUDA, in essence, is like the magic wand that connects software to NVIDIA GPUs. It’s the handshake that enables your AI algorithms to work with the computing power of these graphical beasts. But to NVIDIA’s advantage, CUDA isn’t just any ordinary enchantment, but a closed-source, low-level API that wraps the software around NVIDIA’s GPUs, creating an ecosystem for parallel computing. It’s so potent that even the most formidable competitors such as AMD and Intel struggle to match its finesse.

While other contenders such as Intel and AMD attempt to juggle one or the other, NVIDIA has mastered the art of both. Their GPUs are sleek, powerful, and coveted – and it’s no coincidence that they’ve also laid down the foundations of software that make the most of these machines.

Software companies can’t just waltz in and claim the crown to replace NVIDIA, they lack the hardware prowess. On the flip side, hardware manufacturers can’t wade into the software territory without struggling. This has made CUDA the winning ingredient for NVIDIA in AI.

Undisputed but vulnerable

NVIDIA built CUDA in 2006 with parallel computing for processing on multiple GPUs simultaneously. Earlier, developers were using models like Microsoft’s Direct3D or Linux’s OpenGL for computational purposes on GPUs, but lacked parallel computing capabilities. After the launch of CUDA, businesses began tailoring their strategies to adopt the software. OpenCL by Khronos Group was the only potential competitor released in 2009. But by then all companies had already started leveraging CUDA, leaving no room or need for it.

NVIDIA’s current strategy sounds all great, but there are some major drawbacks in it as well. Though CUDA is a moat for NVIDIA, the company’s pursuit of an upmarket strategy, focusing on high-priced data centre offerings, might let other companies be able to catch up with their software.

Moreover, the market is rife with a GPU shortage that feels almost mythical, but a few are willing to forsake NVIDIA’s wares for alternatives like AMD or Intel. It’s almost as if tech aficionados would rather gnaw on cardboard than consider a GPU from another company.

NVIDIA’s maintenance of its current dominance is rooted in removing the RAM constraints within its consumer grade GPUs. This situation is likely to change as necessity drives the development of software that efficiently exploits consumer-grade GPUs, potentially aided by open-source solutions or offerings from competitors like AMD and Intel. 

Both Intel and AMD stand a chance at challenging NVIDIA’s supremacy, provided they shift away from mimicking NVIDIA’s high-end approach and instead focus on delivering potent, yet cost-effective GPUs, and build open source solutions. Crucially, they should differentiate themselves by avoiding artificial constraints that limit GPU capabilities, which NVIDIA employs to steer users towards their pricier data centre GPUs.

Even after these existing constraints, a lot of developers choose NVIDIA’s consumer grade GPUs over Intel or AMD for ML development. A lot of recent development in these smaller GPUs has led to people shifting to them for deploying models.

There is another competitor coming up

Interestingly, OpenAI’s Triton emerges as a disruptive force against NVIDIA’s closed-source stronghold with CUDA. Triton, taking Meta’s PyTorch 2.0 input via PyTorch Inductor, carves a path by sidestepping NVIDIA’s CUDA libraries and favouring open-source alternatives like CUTLASS.

While CUDA is an accelerated computing mainstay, Triton broadens the horizon. It bridges languages, enabling high-level ones to match the performance of lower-level counterparts. Triton’s legible kernels empower ML researchers, automating memory management and scheduling while proving invaluable for complex operations like Flash Attention.

Triton is currently only being powered on NVIDIA GPUs, the open-source reach might soon extend beyond, marking the advent of a shift. Numerous hardware vendors are set to join the Triton ecosystem, reducing the effort needed to compile for new hardware.

NVIDIA, with all its might, overlooked a critical aspect – usability. This oversight allowed OpenAI and Meta to craft a portable software stack for various hardware, questioning why NVIDIA didn’t simplify CUDA for ML researchers. The absence of their hand in initiatives like Flash Attention raises eyebrows.

NVIDIA has indeed had the upper hand when it comes to product supremacy. But let’s not underestimate the giants of tech. Cloud providers have rolled up their sleeves, designing their own chips that could give NVIDIA’s GPUs a run for their transistors.

Still, all of this is just wishful thinking as of now.

The post NVIDIA’s AI Supremacy is All About CUDA appeared first on AIM.