[Objective]This study aims to address the inefficiency of AI-for-Science tasks caused by the design and implementation challenges of applying the distributed parallel computing strategies to deep learning models,as we...[Objective]This study aims to address the inefficiency of AI-for-Science tasks caused by the design and implementation challenges of applying the distributed parallel computing strategies to deep learning models,as well as their inefficient execution.[Methods]We propose an automatic distributed parallelization method for AI-for-Science tasks,called FlowAware.Based on the AI-for-Science framework JAX,this approach thoroughly analyzes task characteristics,operator structures,and data flow properties of deep learning models.By incorporating cluster topology information,it constructs a search space for distributed parallel computing strategies.Guided by load balancing and communication optimization objectives,FlowAware automatically identifies optimal distributed parallel computing strategies for AI models.[Results]Comparative experiments conducted on both GPU-like accelerator clusters and GPU clusters demonstrated that FlowAware achieves a throughput improvement of up to 7.8×compared to Alpa.[Conclusions]FlowAware effectively enhances the search efficiency of distributed parallel computing strategies for AI models in scientific computing tasks and significantly improves their computational performance.展开更多
The sequential method is easy to integrate with existing large-scale alternating current(AC)power flow solvers and is therefore a common approach for solving the power flow of AC/direct current(DC)hybrid systems.In th...The sequential method is easy to integrate with existing large-scale alternating current(AC)power flow solvers and is therefore a common approach for solving the power flow of AC/direct current(DC)hybrid systems.In this paper,a highperformance graph computing based distributed parallel implementation of the sequential method with an improved initial estimate approach for hybrid AC/DC systems is developed.The proposed approach is capable of speeding up the entire computation process without compromising the accuracy of result.First,the AC/DC network is intuitively represented by a graph and stored in a graph database(GDB)to expedite data processing.Considering the interconnection of AC grids via high-voltage direct current(HVDC)links,the network is subsequently partitioned into independent areas which are naturally fit for distributed power flow analysis.For each area,the fast-decoupled power flow(FDPF)is employed with node-based parallel computing(NPC)and hierarchical parallel computing(HPC)to quickly identify system states.Furthermore,to reduce the alternate iterations in the sequential method,a new decoupled approach is utilized to achieve a good initial estimate for the Newton-Raphson method.With the improved initial estimate,the sequential method can converge in fewer iterations.Consequently,the proposed approach allows for significant reduction in computing time and is able to meet the requirement of the real-time analysis platform for power system.The performance is verified on standard IEEE 300-bus system,extended large-scale systems,and a practical 11119-bus system in China.展开更多
The paper describes the use of invented,developed,and tested in different countries of the high-level spatial grasp model and technology capable of solving important problems in large social systems,which may be repre...The paper describes the use of invented,developed,and tested in different countries of the high-level spatial grasp model and technology capable of solving important problems in large social systems,which may be represented as dynamic,self-evolving and distributed social networks.The approach allows us to find important solutions on a holistic level by spatial navigation and parallel pattern matching of social networks with active self-propagating scenarios represented in a special recursive language.This approach effectively hides inside the distributed and networked language implementation traditional system management routines,often providing hundreds of times shorter and simpler high-level solution code.The paper highlights the demands to efficient simulation of social systems,briefs the technology used,and provides some programming examples for solutions of practical problems.展开更多
Dynamic task assignment and migration are the key technique to load balancing which plays an important role in the achievement of high performance in distributed computing system. In this paper, we describe the design...Dynamic task assignment and migration are the key technique to load balancing which plays an important role in the achievement of high performance in distributed computing system. In this paper, we describe the design and implementation of an online thread scheduling and migration system (S&M) based on a previous work of LWP -MPI. Experimental results show that performance is enhanced.展开更多
CCF Computility is established as a premier platform for industry professionals,academic scholars,and researchers to exchange innovative ideas,methodologies,and technological advancements in the field of distributed c...CCF Computility is established as a premier platform for industry professionals,academic scholars,and researchers to exchange innovative ideas,methodologies,and technological advancements in the field of distributed computing systems.Formerly known as the CCF(China Computer Federation)Annual Conference on Distributed and Parallel Computing Systems(CCF DPCS),CCF Computility has evolved to better address the rapidly growing importance of computility(算力)a new dimension of productive capacity that is driving transformative development across multiple sectors.展开更多
Designing a good preconditioner for accelerating the iterative solution of the three-dimensional multi-group radiation diffusion equations based on a cell-centered finite volume discretization has been the focus of in...Designing a good preconditioner for accelerating the iterative solution of the three-dimensional multi-group radiation diffusion equations based on a cell-centered finite volume discretization has been the focus of intensive research efforts over the past few decades.In the present paper,we develop a physics-wise splitting preconditioning algorithm with selective relaxation and algebraic multigrid subsolves.The spectral distribution and the degree of the minimal polynomial of its rightpreconditioned matrix together with the conditional convergence property of its iteration method are analyzed.Subsequently,we discuss its sequential implementation as well as the two-level parallelization.Lastly,the new preconditioner is applied to the experimental test cases arising from realistic simulations of the hydrodynamic instability during the deceleration phase of a laser-driven spherical implosion to illustrate the numerical robustness,computational efficiency,parallel strong and weak scalabilities,and its competitiveness with some existing monolithic and block preconditioning approaches.展开更多
Concept design is vital important in development of auto-body and it has great effects on later design work.In this paper,a twolevel cross-sectional optimization approach is presented to shorten concept design cycles....Concept design is vital important in development of auto-body and it has great effects on later design work.In this paper,a twolevel cross-sectional optimization approach is presented to shorten concept design cycles.First,an exact structural analysis approach for spatial semi-rigid framed structures,i.e.,the transfer stiffness matrix method proposed in our previous study,is adopted for both static and dynamic analyses of body-in-white(BIW)structure.A two-level cross-sectional optimization approach is then proposed for an automotive BIW lightweight design,and genetic algorithm is used to solve the optimization models.Afterward,an object-oriented MATLAB toolbox,using distributed parallel computing techniques,is developed to promote the concept design of the BIW structure.Finally,relevant numerical examples demonstrate the validity and accuracy of the proposed method.展开更多
Computer clusters with the shared-nothing architecture are the major computing platforms for big data processing and analysis.In cluster computing,data partitioning and sampling are two fundamental strategies to speed...Computer clusters with the shared-nothing architecture are the major computing platforms for big data processing and analysis.In cluster computing,data partitioning and sampling are two fundamental strategies to speed up the computation of big data and increase scalability.In this paper,we present a comprehensive survey of the methods and techniques of data partitioning and sampling with respect to big data processing and analysis.We start with an overview of the mainstream big data frameworks on Hadoop clusters.The basic methods of data partitioning are then discussed including three classical horizontal partitioning schemes:range,hash,and random partitioning.Data partitioning on Hadoop clusters is also discussed with a summary of new strategies for big data partitioning,including the new Random Sample Partition(RSP)distributed model.The classical methods of data sampling are then investigated,including simple random sampling,stratified sampling,and reservoir sampling.Two common methods of big data sampling on computing clusters are also discussed:record-level sampling and blocklevel sampling.Record-level sampling is not as efficient as block-level sampling on big distributed data.On the other hand,block-level sampling on data blocks generated with the classical data partitioning methods does not necessarily produce good representative samples for approximate computing of big data.In this survey,we also summarize the prevailing strategies and related work on sampling-based approximation on Hadoop clusters.We believe that data partitioning and sampling should be considered together to build approximate cluster computing frameworks that are reliable in both the computational and statistical respects.展开更多
We concentrate on the parallel,fully coupled and fully implicit solution of the sequence of 3-by-3 block-structured linear systems arising from the symmetrypreserving finite volume element discretization of the unstea...We concentrate on the parallel,fully coupled and fully implicit solution of the sequence of 3-by-3 block-structured linear systems arising from the symmetrypreserving finite volume element discretization of the unsteady three-temperature radiation diffusion equations in high dimensions.In this article,motivated by[M.J.Gander,S.Loisel,D.B.Szyld,SIAM J.Matrix Anal.Appl.33(2012)653–680]and[S.Nardean,M.Ferronato,A.S.Abushaikha,J.Comput.Phys.442(2021)110513],we aim to develop the additive and multiplicative Schwarz preconditioners subdividing the physical quantities rather than the underlying domain,and consider their sequential and parallel implementations using a simplified explicit decoupling factor approximation and algebraic multigrid subsolves to address such linear systems.Robustness,computational efficiencies and parallel scalabilities of the proposed approaches are numerically tested in a number of representative real-world capsule implosion benchmarks.展开更多
基金supported by the National Key Research and Development Program of China(2023YFB3001501)the National Natural Science Foundation of China(NSFC)(62302133)+3 种基金the Key Research and Development Program of Zhejiang Province(2024C01026)the Yangtze River Delta Project(2023ZY1068)Hangzhou Key Research Plan Project(2024SZD1A02)the GHfund A(202302019816).
摘要[Objective]This study aims to address the inefficiency of AI-for-Science tasks caused by the design and implementation challenges of applying the distributed parallel computing strategies to deep learning models,as well as their inefficient execution.[Methods]We propose an automatic distributed parallelization method for AI-for-Science tasks,called FlowAware.Based on the AI-for-Science framework JAX,this approach thoroughly analyzes task characteristics,operator structures,and data flow properties of deep learning models.By incorporating cluster topology information,it constructs a search space for distributed parallel computing strategies.Guided by load balancing and communication optimization objectives,FlowAware automatically identifies optimal distributed parallel computing strategies for AI models.[Results]Comparative experiments conducted on both GPU-like accelerator clusters and GPU clusters demonstrated that FlowAware achieves a throughput improvement of up to 7.8×compared to Alpa.[Conclusions]FlowAware effectively enhances the search efficiency of distributed parallel computing strategies for AI models in scientific computing tasks and significantly improves their computational performance.
基金supported by the State Grid Corporation Technology Project(No.5455HJ180022)。
摘要The sequential method is easy to integrate with existing large-scale alternating current(AC)power flow solvers and is therefore a common approach for solving the power flow of AC/direct current(DC)hybrid systems.In this paper,a highperformance graph computing based distributed parallel implementation of the sequential method with an improved initial estimate approach for hybrid AC/DC systems is developed.The proposed approach is capable of speeding up the entire computation process without compromising the accuracy of result.First,the AC/DC network is intuitively represented by a graph and stored in a graph database(GDB)to expedite data processing.Considering the interconnection of AC grids via high-voltage direct current(HVDC)links,the network is subsequently partitioned into independent areas which are naturally fit for distributed power flow analysis.For each area,the fast-decoupled power flow(FDPF)is employed with node-based parallel computing(NPC)and hierarchical parallel computing(HPC)to quickly identify system states.Furthermore,to reduce the alternate iterations in the sequential method,a new decoupled approach is utilized to achieve a good initial estimate for the Newton-Raphson method.With the improved initial estimate,the sequential method can converge in fewer iterations.Consequently,the proposed approach allows for significant reduction in computing time and is able to meet the requirement of the real-time analysis platform for power system.The performance is verified on standard IEEE 300-bus system,extended large-scale systems,and a practical 11119-bus system in China.
摘要The paper describes the use of invented,developed,and tested in different countries of the high-level spatial grasp model and technology capable of solving important problems in large social systems,which may be represented as dynamic,self-evolving and distributed social networks.The approach allows us to find important solutions on a holistic level by spatial navigation and parallel pattern matching of social networks with active self-propagating scenarios represented in a special recursive language.This approach effectively hides inside the distributed and networked language implementation traditional system management routines,often providing hundreds of times shorter and simpler high-level solution code.The paper highlights the demands to efficient simulation of social systems,briefs the technology used,and provides some programming examples for solutions of practical problems.
摘要Dynamic task assignment and migration are the key technique to load balancing which plays an important role in the achievement of high performance in distributed computing system. In this paper, we describe the design and implementation of an online thread scheduling and migration system (S&M) based on a previous work of LWP -MPI. Experimental results show that performance is enhanced.
摘要CCF Computility is established as a premier platform for industry professionals,academic scholars,and researchers to exchange innovative ideas,methodologies,and technological advancements in the field of distributed computing systems.Formerly known as the CCF(China Computer Federation)Annual Conference on Distributed and Parallel Computing Systems(CCF DPCS),CCF Computility has evolved to better address the rapidly growing importance of computility(算力)a new dimension of productive capacity that is driving transformative development across multiple sectors.
基金supported in part by National Natural Science Foundation of China(12371373)Hunan National Applied Mathematics Center(2020ZYT003)+1 种基金Research Foundation of Education Bureau of Hunan(21B0162)Shandong Provincial Natural Science Foundation(ZR2020MA046).
摘要Designing a good preconditioner for accelerating the iterative solution of the three-dimensional multi-group radiation diffusion equations based on a cell-centered finite volume discretization has been the focus of intensive research efforts over the past few decades.In the present paper,we develop a physics-wise splitting preconditioning algorithm with selective relaxation and algebraic multigrid subsolves.The spectral distribution and the degree of the minimal polynomial of its rightpreconditioned matrix together with the conditional convergence property of its iteration method are analyzed.Subsequently,we discuss its sequential implementation as well as the two-level parallelization.Lastly,the new preconditioner is applied to the experimental test cases arising from realistic simulations of the hydrodynamic instability during the deceleration phase of a laser-driven spherical implosion to illustrate the numerical robustness,computational efficiency,parallel strong and weak scalabilities,and its competitiveness with some existing monolithic and block preconditioning approaches.
基金The authors acknowledge financial support from the National Natural Science Foundation of China(No.51475152).
摘要Concept design is vital important in development of auto-body and it has great effects on later design work.In this paper,a twolevel cross-sectional optimization approach is presented to shorten concept design cycles.First,an exact structural analysis approach for spatial semi-rigid framed structures,i.e.,the transfer stiffness matrix method proposed in our previous study,is adopted for both static and dynamic analyses of body-in-white(BIW)structure.A two-level cross-sectional optimization approach is then proposed for an automotive BIW lightweight design,and genetic algorithm is used to solve the optimization models.Afterward,an object-oriented MATLAB toolbox,using distributed parallel computing techniques,is developed to promote the concept design of the BIW structure.Finally,relevant numerical examples demonstrate the validity and accuracy of the proposed method.
基金Supported in part by the National Natural Science Foundation of China(No.61972261)the National Key R&D Program of China(No.2017YFC0822604-2)
摘要Computer clusters with the shared-nothing architecture are the major computing platforms for big data processing and analysis.In cluster computing,data partitioning and sampling are two fundamental strategies to speed up the computation of big data and increase scalability.In this paper,we present a comprehensive survey of the methods and techniques of data partitioning and sampling with respect to big data processing and analysis.We start with an overview of the mainstream big data frameworks on Hadoop clusters.The basic methods of data partitioning are then discussed including three classical horizontal partitioning schemes:range,hash,and random partitioning.Data partitioning on Hadoop clusters is also discussed with a summary of new strategies for big data partitioning,including the new Random Sample Partition(RSP)distributed model.The classical methods of data sampling are then investigated,including simple random sampling,stratified sampling,and reservoir sampling.Two common methods of big data sampling on computing clusters are also discussed:record-level sampling and blocklevel sampling.Record-level sampling is not as efficient as block-level sampling on big distributed data.On the other hand,block-level sampling on data blocks generated with the classical data partitioning methods does not necessarily produce good representative samples for approximate computing of big data.In this survey,we also summarize the prevailing strategies and related work on sampling-based approximation on Hadoop clusters.We believe that data partitioning and sampling should be considered together to build approximate cluster computing frameworks that are reliable in both the computational and statistical respects.
基金financially supported by Hunan National Applied Mathematics Center(2020ZYT003)National Natural Science Foundation of China(11971414,62102167)+1 种基金Research Foundation of Education Bureau of Hunan(21B0162)Guangdong Basic and Applied Basic Research Foundation(2020A1515110364).
摘要We concentrate on the parallel,fully coupled and fully implicit solution of the sequence of 3-by-3 block-structured linear systems arising from the symmetrypreserving finite volume element discretization of the unsteady three-temperature radiation diffusion equations in high dimensions.In this article,motivated by[M.J.Gander,S.Loisel,D.B.Szyld,SIAM J.Matrix Anal.Appl.33(2012)653–680]and[S.Nardean,M.Ferronato,A.S.Abushaikha,J.Comput.Phys.442(2021)110513],we aim to develop the additive and multiplicative Schwarz preconditioners subdividing the physical quantities rather than the underlying domain,and consider their sequential and parallel implementations using a simplified explicit decoupling factor approximation and algebraic multigrid subsolves to address such linear systems.Robustness,computational efficiencies and parallel scalabilities of the proposed approaches are numerically tested in a number of representative real-world capsule implosion benchmarks.