VMBlog – MBX 2023 Prediction: Acceleration of ARM Adoption for Server and Edge Applications
February 2, 2023 — Bill ConradesDirector of Sales Engineering
For years, ARM processors have powered the vast majority of the world’s smartphones, largely because their low power consumption helps deliver all-day battery life even for gamers and users with other compute-intensive workloads.
Today, performance improvements and other advances are expanding the use of ARM technology to newer uses like laptops, servers and edge computing applications, offering benefits ranging from energy efficiency and low price points to optimized support for cloud computing, artificial intelligence and machine learning.
The shift to the use of ARM architecture in enterprise computing vastly accelerated in 2020, when Apple announced that it was transitioning a portion of its Mac computers from the Intel chips it had used since 2006 to its own ARM-based M1 system-on-a-chip (SOC). The trend has been steadily accelerating ever since. Here are several examples.
1 – ARM arrives in the data center: After a series of false starts that led to the shutdown of companies like Calxeda a decade ago, Ampere Computing is achieving substantial success in providing an ARM-based, cloud-focused server architecture with its Altra® SOC solutions featuring the world’s only 128-core cloud native processor. The low TCO makes Ampere’s Altra reference platforms well-suited for edge computing with heavy workloads, AI/ML and GPU workloads, and applications demanding high core counts, flexible storage and peak memory.
Major ODMs like Supermicro, Gigabyte, Foxconn, ADLINK and Wiwynn have added Ampere Altra-based server offerings to their portfolios in the last year. So have Google Cloud, HPE and Microsoft, which has introduced Azure VMs with Ampere Altra ARM-based processors.
This clearly signals the dawn of a new infrastructure model, with a 2022 study by Trendforce predicting that ARM architecture will have a 22% share of data-center servers by 2025 led by cloud data centers.
2 – Netflix tests ARM platform to double server bandwidth: In 2021, in an effort to boost the TLS-encrypted bandwidth of its video servers from 200 Gbps to 400 Gbps, Netflix compared servers equipped with AMD EPYC 7002 (Rome), Intel Xeon Platinum 8352V (Ice Lake) and Ampere Altra Q80-30 processors. The AMD configuration topped the field at 380 Gbps when TLS encryption was offloaded to optimize the performance and avoid hardware bottlenecks, but Altra’s ARM-based solution finished a close second at 320 Gbps.
Netflix testers theorized that the slower performance was caused by a PCIe-specific problem because they noticed low processor utilization and saturated NICs with lots of output drops. Considering benefits like energy and cost savings, ARM will likely be a strong contender for content delivery networks going forward.
3 – ARM CPUs drive AI at the edge: NVIDIA Jetson models combine an ARM architecture central processing unit and an NVIDIA GPU within a SOC designed by NVIDIA. The Jetson platforms take advantage of ARM’s high-performance and low power to run artificial intelligence and machine learning workloads at the edge. This is fueling the development of new edge-based inferencing solutions in areas ranging from medical, safety and security applications to robotics, industrial automation and more.
One example is MBX Systems’ Kori, the first mobile cart that can be outfitted with different cameras, workstations and peripherals to accommodate the needs of different computer vision applications. Preferred compute options currently utilize the NVIDIA Jetson Xavier NX with plans to transition to the next-generation Jetson Orin NX for single camera inferencing and the Jetson Orin AGX for inferencing multiple camera streams. MBX will have a dual camera AI demo running on the AGX Orin development kit at the MD&M West trade show in Anaheim, CA, in February.
The cart also uses an ARM-based Raspberry Pi module to power the 157 LEDs that can be individually programmed with different color and sequencing patterns to indicate usage status, alerts and other functions.
In hospital environments, for instance, Kori can be used to deploy solutions that can detect patient falls, screen body temperatures, facilitate patient-healthcare provider communication and enable remote live surgical theater collaborations. The low power consumption made possible by the embedded ARM CPU is particularly useful in hospital settings because it allows inferencing to be performed against the camera stream even if the cart is unplugged and running on battery.
ARM adoption beyond smartphones is just beginning. As this is written, for example, NVIDIA is launching an industrial-grade edge AI computing platform called NVIDIA IGX that will achieve a new level of performance per dollar and thereby make large-scale edge inferencing a reality. Potential applications range from bringing AI to entire traffic systems, smart cities and autonomous factories to medical uses such as robotic surgery controls, endoscopy, diagnostic imaging, radiation therapy, and microscopy.
The trend is clear. ARM is entering the mainstream, opening new opportunities for energy-efficient, performant, lower cost of ownership product development that will continue to transform the technology landscape. We’ll all be watching.
Read the original article at vmblog.com
Medical AI: From Cloud to Edge Inferencing
MBX’s High-Touch Manufacturing & Logistics Services Fuel Record Revenue
CDN / Video Streaming
Custom Compute Solutions for Medical Device Companies
Custom Industrial Design Solutions
CDN / Video Streaming
What is the IoMT? Everything You Need to Know About Hyper-Connected Medical Devices