In our modern age, our lives have become intertwined with smart devices that have the ability to carry out a multitude of tasks. From voice commands to tracking our health, these devices have become indispensable. However, when it comes to AI capabilities on portable devices, there has been a significant stumbling block – memory. Large language models, which have revolutionized communication between humans and technology, require a vast amount of memory. This poses a problem for commonly used smartphones that have limited memory capacities. Apple, recognizing this issue, has recently announced a breakthrough method to overcome this limitation.
In a paper published on the pre-print server arXiv, Apple unveiled their groundbreaking method for running powerful AI systems on smart devices with limited memory. By utilizing transfers of data between flash memory and DRAM, Apple’s researchers have found a way to enable a device to run AI programs that are twice the size of its DRAM capacity. Not only that, but they have managed to speed up CPU operations by up to 500% and GPU processes by up to 25 times the current approaches.
The researchers at Apple employed two key techniques to achieve their breakthrough. The first technique, called windowing, reduces the amount of data that needs to be exchanged between flash memory and RAM. This is accomplished by reusing results from recent calculations, minimizing IO requests, and saving both time and energy. The second technique, known as row column bundling, enhances efficiency by processing larger chunks of data at a time from flash memory. Together, these two techniques significantly reduce the data load and optimize memory usage.
Expanding Applicability and Accessibility
The significance of Apple’s breakthrough in AI memory optimization cannot be understated. It opens up opportunities for deploying advanced large language models (LLMs) in resource-limited environments, thereby expanding their applicability and accessibility. Previously, the limitations of memory capacity on portable devices hindered the deployment of more powerful AI systems. Now, with Apple’s method, the potential for LLMs to revolutionize various industries increases substantially.
Apple’s Other Recent Breakthrough
In addition to their breakthrough in AI memory optimization, Apple has also made strides in avatar creation. They have developed a program called HUGS (Human Gaussian Splats) that can create animated avatars from just a few seconds of video captured from a single lens. Unlike current avatar creation programs that require multiple camera views, Apple’s program can generate realistic dancing avatars in as little as 30 minutes. This is a significant improvement compared to the two days that conventional approaches would take.
Apple’s advancements in AI memory optimization and avatar creation highlight their commitment to pushing the boundaries of what is possible on portable devices. As technology continues to evolve, we can expect further breakthroughs that enhance the capabilities and functionalities of our smart devices. The power of large language models combined with Apple’s innovative solutions will undoubtedly shape the future of AI on portable devices, making our interactions with technology more seamless and efficient.
The use of large language models and natural language processing has revolutionized the way we communicate with technology and the capabilities of our smart devices. However, the limitation of memory on portable devices has hindered the deployment of powerful AI systems. Apple’s breakthrough in AI memory optimization addresses this issue and opens up new possibilities for the utilization of large language models. With their innovative techniques, Apple has not only increased the efficiency of memory usage but also expanded the applicability and accessibility of advanced LLMs. Furthermore, their advancements in avatar creation demonstrate their dedication to pushing the boundaries of what can be achieved on portable devices. As we look to the future, it is exciting to imagine the further advancements and possibilities that will undoubtedly emerge in the world of AI on portable devices.