Current Issue
lssue 3, 2025
Editors-in-Chief
Yunhe Pan, Aiguo Fei
Advisor Xicheng Lu
ISSN 2095-9184
CN 33-1389/TP
IF (2023 JCR) 2.7
| Submit Now |
- Current Issue
- Cover Articles
- Archive
- Virtual Issues
- Online First
Volume 26 Issue 3，2025
Regular Papers
Training large-scale language models with limited GPU memory: a survey Enhanced Publication
Abstract：Large-scale models have gained significant attention in a wide range of fields, such as computer vision and natural language processing, due to their effectiveness across various applications. However, a notable hurdle in training these large-scale models is the limited memory capacity of graphics processing units (GPUs). In this paper, we present a comprehensive survey focused on training large-scale models with limited GPU memory. The exploration commences by scrutinizing the factors that contribute to the consumption of GPU memory during the training process, namely model parameters, model states, and model activations. Following this analysis, we present an in-depth overview of the relevant research work that addresses these aspects individually. Finally, the paper concludes by presenting an outlook on the future of memory optimization in training large-scale language models, emphasizing the necessity for continued research and innovation in this area. This survey serves as a valuable resource for researchers and practitioners keen on comprehending the challenges and advancements in training large-scale language models with limited GPU memory.Keywords：Training techniques;Memory optimization;Model parameters;Model states;Model activations122|30|0Updated：2025-04-03Regular Papers
A review of flexible job shop scheduling problems considering transportation vehicles Enhanced Publication
Abstract：The flexible job shop scheduling problem for processing machines and transportation vehicles (FJSP_PT) has garnered significant attention from academia and industry. Due to the inclusion of transportation vehicle scheduling in the scheduling problem of flexible manufacturing systems, solving FJSP_PT becomes more challenging and significantly more practically relevant compared to the flexible job shop scheduling problem. We summarize the assumptions, constraints, objective functions, and benchmarks of FJSP_PT. Then, statistical analysis is conducted on the literature up to 2023, including journals, number of articles published each year, and solution algorithms. We analyze recent literature on FJSP_PT, categorizing it based on algorithms into exact algorithms, heuristic algorithms, meta-heuristic algorithms, and swarm intelligence based algorithms. Finally, the research trends and challenges faced by FJSP_PT are summarized.Keywords：Flexible manufacturing system;Transportation vehicle;Processing machine;Integrated scheduling62|21|0Updated：2025-04-03Regular Papers
DRMSpell: dynamically reweighting multimodality for Chinese spelling correction Cover Article Enhanced Publication
Abstract：Chinese spelling correction (CSC) is a task that aims to detect and correct the spelling errors that may occur in Chinese texts. However, the Chinese language exhibits a high degree of complexity, characterized by the presence of multiple phonetic representations known as pinyin, which possess distinct tonal variations that can correspond to various characters. Given the complexity inherent in the Chinese language, the CSC task becomes imperative for ensuring the accuracy and clarity of written communication. Recent research has included external knowledge into the model using phonological and visual modalities. However, these methods do not effectively target the utilization of modality information to address the different types of errors. In this paper, we propose a multimodal pretrained language model called DRMSpell for CSC, which takes into consideration the interaction between the modalities. A dynamically reweighting multimodality (DRM) module is introduced to reweight various modalities for obtaining more multimodal information. To fully use the multimodal information obtained and to further strengthen the model, an independent-modality masking strategy (IMS) is proposed to independently mask three modalities of a token in the pretraining stage. Our method achieves state-of-the-art performance on most metrics constituting widely used benchmarks. The findings of the experiments demonstrate that our method is capable of modeling the interactive information between modalities and is also robust to incorrect modal information.Keywords：Chinese spelling correction;Multimodality;Masking strategy33|24|0Updated：2025-04-03Regular Papers
An intelligent mesh-smoothing method with graph neural networks Enhanced Publication
Abstract：In computational fluid dynamics (CFD), mesh-smoothing methods are widely used to refine the mesh quality for achieving high-precision numerical simulations. Specifically, optimization-based smoothing is used for high-quality mesh smoothing, but it incurs significant computational overhead. Pioneer works have improved its smoothing efficiency by adopting supervised learning to learn smoothing methods from high-quality meshes. However, they pose difficulties in smoothing the mesh nodes with varying degrees and require data augmentation to address the node input sequence problem. Additionally, the required labeled high-quality meshes further limit the applicability of the proposed method. In this paper, we present graph-based smoothing mesh net (GMSNet), a lightweight neural network model for intelligent mesh smoothing. GMSNet adopts graph neural networks (GNNs) to extract features of the node’s neighbors and outputs the optimal node position. During smoothing, we also introduce a fault-tolerance mechanism to prevent GMSNet from generating negative volume elements. With a lightweight model, GMSNet can effectively smooth mesh nodes with varying degrees and remain unaffected by the order of input data. A novel loss function, MetricLoss, is developed to eliminate the need for high-quality meshes, which provides stable and rapid convergence during training. We compare GMSNet with commonly used mesh-smoothing methods on two-dimensional (2D) triangle meshes. Experimental results show that GMSNet achieves outstanding mesh-smoothing performances with 5% of the model parameters compared to the previous model, but offers a speedup of 13.56 times over the optimization-based smoothing.Keywords：Unstructured mesh;Mesh smoothing;Graph neural network;Optimization-based smoothing39|25|0Updated：2025-04-03Regular Papers
Significance extraction based on data augmentation for reinforcement learning Enhanced Publication
Abstract：Deep reinforcement learning has shown remarkable capabilities in visual tasks, but it does not have a good generalization ability in the context of interference signals in the input images; this approach is therefore hard to be applied to trained agents in a new environment. To enable agents to distinguish between noise signals and important pixels in images, data augmentation techniques and the establishment of auxiliary networks are proven effective solutions. We introduce a novel algorithm, namely, saliency-extracted Q-value by augmentation (SEQA), which encourages the agent to explore unknown states more comprehensively and focus its attention on important information. Specifically, SEQA masks out interfering features and extracts salient features and then updates the mask decoder network with critic losses to encourage the agent to focus on important features and make correct decisions. We evaluate our algorithm on the DeepMind Control generalization benchmark (DMControl-GB), and the experimental results show that our algorithm greatly improves training efficiency and stability. Meanwhile, our algorithm is superior to state-of-the-art reinforcement learning methods in terms of sample efficiency and generalization in most DMControl-GB tasks.Keywords：Deep reinforcement learning;Visual tasks;Generalization;Data augmentation;Significance;DeepMind Control generalization benchmark30|22|0Updated：2025-04-03Regular Papers
Distributed multi-target tracking with labeled multi-Bernoulli filter considering efficient label matching Enhanced Publication
Abstract：We propose a distributed labeled multi-Bernoulli (LMB) filter based on an efficient label matching method. Conventional distributed LMB filter fusion has the premise that the labels among local densities have already been matched. However, considering that the label space of each local posterior is independent, such a premise is not practical in many applications. To achieve distributed fusion practically, we propose an efficient label matching method derived from the divergence of arithmetic average (AA) mechanism, and subsequently label-wise LMB filter fusion is performed according to the matching results. Compared with existing label matching methods, this proposed method shows higher performance, especially in low detection probability scenarios. Moreover, to guarantee the consistency and completeness of the fusion outcome, the overall fusion procedure is designed into the following four stages: pre-fusion, label determination, posterior complement, and uniqueness check. The performance of the proposed label matching distributed LMB filter fusion is demonstrated in a challenging nonlinear bearings-only multi-target tracking (MTT) scenario.Keywords：Distributed multi-sensor multi-target tracking;Labeled multi-Bernoulli filter;Arithmetic average fusion;Label matching26|17|0Updated：2025-04-03
SEE MORE
0 2 Hybrid-augmented intelligence: collaboration and cognition
0 3 Applications of artificial intelligence in intelligent manufacturing: a review*
0 5 Cross-media analysis and reasoning: advances and directions
0 6 Recent advances in efficient computation of deep convolutional neural networks
0 7 Challenges and opportunities: from big data to knowledge in AI 2.0
0 8 Current trends in the development of intelligent unmanned autonomous systems
10 Real-time road traffic state prediction based on ARIMA and Kalman filter
SEE MORE
Videos
-
00:02:512023 Issue 1 | Scalability and efficiency challenges for the exascale supercomputing system: practice of a parallel supporting environment on the Sunway exascale prototype system
2023-12-30Play Total： 22 - 00:02:30
2023 Issue 6 | Model division multiple access for semantic communications
2023-12-30Play Total： 11 - 00:02:15
2022 Issue 10 | Discussion on a new paradigm of endogenous security towards 6G networks
2023-12-30Play Total： 0 - 00:02:22
2022 Issue 12 | Technology trends in large-scale high-efficiency network computing
2023-12-30Play Total： 2 - 00:02:48
2022 Issue 6 | Self-deployed execution environment for high performance computing
2022-08-03Play Total： 7 - 00:02:24
2022 Issue 2 | A full-process intelligent trial system for smart court
2022-05-17Play Total： 7 - 00:02:37
2022 Issue 3 | Automatic protocol reverse engineering for industrial control systems with dynamic taint analysis
2022-05-17Play Total： 4 - 00:05:36
P1 Speech by Academician Baoyan Duan
2022-04-17Play Total： 9 - 00:02:27
P2 Speech by Professor Min Sheng, Xidian University
2022-04-17Play Total： 6 - 00:02:37
P3 Speech by Professor Yunsong Li, Xidian University
2022-04-17Play Total： 10
SEE MORE
- Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
- Tel：+86-571-88273162 Email：fitee@zju.edu.cn
- Copyright © 2000 - 2025, Zhejiang University Press 京ICP备09064830号-19
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
0