Scaling up neural networks for deep learning. Free Technical Library News

NEWS OF SCIENCE AND TECHNOLOGY, NOVELTY IN ELECTRONICS
Free technical library / timeline

Scaling Up Neural Networks for Deep Learning

15.10.2016

Fujitsu announced the development of a technology that optimizes the use of graphics processing unit (GPU) internal memory to support the increasing scale of neural networks used to improve the accuracy of machine learning.

One method for scaling up deep learning is to distribute a single neural network model across multiple machines and run the computations in parallel. But the large amount of data that must be transferred between computers creates traffic jams, significantly reducing the speed of tasks, Fujitsu explained. In order to take advantage of the full potential of GPUs for high-speed computing, data must be stored in the internal memory of the processors themselves. However, this amount is usually less than the memory capacity of conventional computers, which limits the ability to scale up neural networks.

Fujitsu has developed technology to optimize memory usage and scale up neural networks for single GPU computing. The novelty does not use methods of parallel organization of work, which significantly reduce the speed of reading. The new technology reduces the amount of memory required by reusing resources: it allows you to independently perform calculations to create intermediate error data from weighted data and calculations to create weighted data from intermediate data. When training begins, the structure of each layer of the neural network is analyzed and the calculation order is changed so that the memory area, which contains more data, can be reused, the company noted.

The company used the new technology as part of the Caffe open source deep learning platform, measuring the consumption of internal GPU memory. After starting training, the technology analyzes the structure of the neural network and optimizes the order of execution of calculations and the location of data in memory so that the free memory area can be reused. When used in conjunction with AlexNet and VGGNet, image recognition neural networks used for scientific research, Fujitsu's new technology has increased the scale of the neural network by up to 2 times compared to previous counterparts. As such, the amount of internal GPU memory used has been reduced by more than 40%, according to the company.

Plans to use the new technology from March 31, 2017 as part of the Human Centric AI Zinrai artificial intelligence project. In addition, the company plans to use this technology along with previously unveiled technology for deep learning data processing using GPU parallelization.

<< Back: Cats can heal people 16.10.2016

>> Forward: Bees have emotions and mood swings 15.10.2016

Latest news of science and technology, new electronics:

The existence of an entropy rule for quantum entanglement has been proven 09.05.2024

Quantum mechanics continues to amaze us with its mysterious phenomena and unexpected discoveries. Recently, Bartosz Regula from the RIKEN Center for Quantum Computing and Ludovico Lamy from the University of Amsterdam presented a new discovery that concerns quantum entanglement and its relation to entropy. Quantum entanglement plays an important role in modern quantum information science and technology. However, the complexity of its structure makes understanding and managing it challenging. Regulus and Lamy's discovery shows that quantum entanglement follows an entropy rule similar to that for classical systems. This discovery opens new perspectives in the field of quantum information science and technology, deepening our understanding of quantum entanglement and its connection to thermodynamics. The results of the study indicate the possibility of reversibility of entanglement transformations, which could greatly simplify their use in various quantum technologies. Opening a new rule ... >>

Mini air conditioner Sony Reon Pocket 5 09.05.2024

Summer is a time for relaxation and travel, but often the heat can turn this time into an unbearable torment. Meet a new product from Sony - the Reon Pocket 5 mini-air conditioner, which promises to make summer more comfortable for its users. Sony has introduced a unique device - the Reon Pocket 5 mini-conditioner, which provides body cooling on hot days. With it, users can enjoy coolness anytime, anywhere by simply wearing it around their neck. This mini air conditioner is equipped with automatic adjustment of operating modes, as well as temperature and humidity sensors. Thanks to innovative technologies, Reon Pocket 5 adjusts its operation depending on the user's activity and environmental conditions. Users can easily adjust the temperature using a dedicated mobile app connected via Bluetooth. Additionally, specially designed T-shirts and shorts are available for convenience, to which a mini air conditioner can be attached. The device can oh ... >>

Energy from space for Starship 08.05.2024

Producing solar energy in space is becoming more feasible with the advent of new technologies and the development of space programs. The head of the startup Virtus Solis shared his vision of using SpaceX's Starship to create orbital power plants capable of powering the Earth. Startup Virtus Solis has unveiled an ambitious project to create orbital power plants using SpaceX's Starship. This idea could significantly change the field of solar energy production, making it more accessible and cheaper. The core of the startup's plan is to reduce the cost of launching satellites into space using Starship. This technological breakthrough is expected to make solar energy production in space more competitive with traditional energy sources. Virtual Solis plans to build large photovoltaic panels in orbit, using Starship to deliver the necessary equipment. However, one of the key challenges ... >>

New method for creating powerful batteries 08.05.2024

With the development of technology and the expanding use of electronics, the issue of creating efficient and safe energy sources is becoming increasingly urgent. Researchers at the University of Queensland have unveiled a new approach to creating high-power zinc-based batteries that could change the landscape of the energy industry. One of the main problems with traditional water-based rechargeable batteries was their low voltage, which limited their use in modern devices. But thanks to a new method developed by scientists, this drawback has been successfully overcome. As part of their research, scientists turned to a special organic compound - catechol. It turned out to be an important component that can improve battery stability and increase its efficiency. This approach has led to a significant increase in the voltage of zinc-ion batteries, making them more competitive. According to scientists, such batteries have several advantages. They have b ... >>

Alcohol content of warm beer 07.05.2024

Beer, as one of the most common alcoholic drinks, has its own unique taste, which can change depending on the temperature of consumption. A new study by an international team of scientists has found that beer temperature has a significant impact on the perception of alcoholic taste. The study, led by materials scientist Lei Jiang, found that at different temperatures, ethanol and water molecules form different types of clusters, which affects the perception of alcoholic taste. At low temperatures, more pyramid-like clusters form, which reduces the pungency of the "ethanol" taste and makes the drink taste less alcoholic. On the contrary, as the temperature increases, the clusters become more chain-like, resulting in a more pronounced alcoholic taste. This explains why the taste of some alcoholic drinks, such as baijiu, can change depending on temperature. The data obtained opens up new prospects for beverage manufacturers, ... >>

Random news from the Archive

Ecological synthesis of urea 06.10.2023

Scientists from the University of Queensland (QUT) have unveiled an environmentally friendly method for synthesizing the vital agricultural fertilizer urea. This method allows the production of urea at room temperature with minimal energy consumption, in contrast to the traditional synthesis process using large amounts of energy.

Dr Junxian Liu, the study's lead author, together with Professor Yuantong Gu and Associate Professor Liangzhi Kou from the School of Engineering, Medicine and Technology, have discovered an effective way to create urea, an important nitrogen fertilizer that supports about 27% of global agriculture. Urea is also used in pharmaceuticals, cosmetics and plastics production.

Unlike the traditional industrial process, which involves high temperatures and pressure, the new method offers urea synthesis at room temperature and atmospheric pressure. This is achieved through a chemical reaction between nitrogen and carbon monoxide using a graphene-based catalyst.

This approach significantly reduces energy consumption compared to traditional methods, making it a promising advance for urea and fertilizer production in general.

See full Archive of science and technology news, new electronics

All languages of this page

Home page | Library | Articles | Website map | Site Reviews