万彩网·首页欢迎您

pisco_log

Open Journal Systems

User

Language

Journal Content

Font Size

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z All

Geological and Mining Engineering

Geological and Mining Engineering (GME), published by Scineer Publishing PTE. LTD, is an open-access journal. GME is devoted to publish geological and mining engineering related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on geological and mining engineering throughout the world. GME welcomes all original

ISSN: 3029-1887(P)

3029-2034(O)

View Journal Current Issue Register

Food Safety and Quality

Food Safety and Quality (FSQ), published by Scineer Publishing PTE. LTD, is an open-access journal. FSQ is devoted to publish food safety and quality related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on food safety and quality throughout the world. FSQ welcomes all original manuscripts that do not be published in any other journals.

ISSN: 3029-2115(P)

3029-2123(O)

View Journal Current Issue Register

Hydraulic Engineering and Power Research

Hydraulic Engineering and Power Research (HEPR), published by Scineer Publishing PTE. LTD, is an open-access journal. HEPR is devoted to publish hydraulic engineering and power related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on hydraulic engineering and power throughout the world. HEPR welcomes all original manuscripts that do not be published in any other journals.Â

Â Â Â

Â Â Â Â ISSN: 3029-2158(P)

Â Â Â Â Â Â Â Â Â 3029-2166(O)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Medical and Health Research

Medical and Health Research (MHR), published by Scineer Publishing PTE. LTD, is an open-access journal. MHR is devoted to publish medicine related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on medicine throughout the world. MHR welcomes all original manuscripts that do not be published in any other journals.Â

Â Â Â

Â Â Â Â ISSN: 3029-2190(P)

Â Â Â Â Â Â Â Â Â 3029-2204(O)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Practical Medical Research

Practical Medical Research (PMR), published by Scineer Publishing PTE. LTD, is an open-access journal. PMR is devoted to publish medicine related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on medicine throughout the world. PMR welcomes all original manuscripts that do not be published in any other journals.Â

Â Â Â Â

Â Â Â Â ISSN: 3029-2085(P)

Â Â Â Â Â Â Â Â Â Â 3029-2093(O)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Forum on Research and Innovation Management

Forum on Research and Innovation Management (FRIM), published by Scineer Publishing PTE. LTD, is an open-access journal. FRIM is devoted to publish research and innovation management related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on research and innovation management throughout the world. FRIM welcomes all original manuscripts that do not be published in any other journals.Â

Â Â Â Â

Â Â Â Â ISSN: 3029-1933(P)

Â Â Â Â Â Â Â Â Â 3029-1984(O)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Construction Safety Technology and Management

Construction Safety Technology and Management (CSTM), published by Scineer Publishing PTE. LTD, is an open-access journal. CSTM is devoted to publish construction safety technology and management related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on construction safety technology and management throughout the world. CSTM welcomes all original manuscripts that do not be published in any other journals.Â

Â Â Â

Â Â Â Â Â ISSN: 3029-2069(P)

Â Â Â Â Â Â Â Â Â Â Â 3029-2077(O)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Industrial Technology Research

Industrial Technology Research (ITR), published by Scineer Publishing PTE. LTD, is an open-access journal. ITR is devoted to publish industrial technology related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on industrial technology throughout the world. ITR welcomes all original manuscripts that do not be published in any other journals.

Â Â Â Â

Â Â Â Â ISSN: 3029-1895(P)

Â Â Â Â Â Â Â Â Â Â 3029-2026(O)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Environmental Protection and Health

Environmental Protection and Health (EPH), published by Scineer Publishing PTE. LTD, is an open-access journal. EPH is devoted to publish environmental protection and health related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on environmental protection and health throughout the world. EPH welcomes all original manuscripts that do not be published in any other journals.Â

Â Â

Â Â Â Â ISSN: 3029-1909(P)

Â Â Â Â Â Â Â Â Â 3029-2018(O)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Artificial Intelligence Technology Research

Artificial Intelligence Technology Research (AITR), published by Scineer Publishing PTE. LTD, is an open-access journal. AITR is devoted to publish artificial intelligence technology related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on artificial intelligence technology throughout the world. AITR welcomes all original manuscripts that do not be published in any other journals.Â

Â Â Â Â ISSN: 3029-1968(O)

Â Â Â Â Â Â Â Â Â 3029-195X(P)
Â Â Â Â Â

Â Â

View Journal Current Issue Register

Curriculum Learning and Exploration

Curriculum Learning and Exploration (CLE), published by Scineer Publishing PTE. LTD, is an open-access journal. CLE is devoted to publish curriculum related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on curriculum throughout the world. CLE welcomes all original manuscripts that do not be published in any other journals.Â

Â Â Â Â ISSN: 3029-1976(O)

Â Â Â Â Â Â Â Â Â 3029-1941(P)
Â Â Â Â Â Â Â

View Journal Current Issue Register

Evaluation of Educational Research

Evaluation of Educational Research (EER), published by Scineer Publishing PTE. LTD, is an open-access journal. EER is devoted to publish education related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on education throughout the world. EER welcomes all original manuscripts that do not be published in any other journals.

Â Â Â Â ISSN: 3029-1992(O)

Â Â Â Â Â Â Â Â Â 3029-1925(P)
Â Â Â Â Â Â Â

View Journal Current Issue Register

World Education Forum

World Education Forum (WEF), published by Scineer Publishing PTE. LTD, is an open-access journal. WEF is devoted to publish education related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on education throughout the world. WEF welcomes all original manuscripts that do not be published in any other journals.

Â Â Â Â ISSN: 3029-214X(O)

Â Â Â Â Â Â Â Â Â 3029-2131(P)

Â Â Â Â Â Â Â

View Journal Current Issue Register

New Explorations in Education and Teaching

New Explorations in Education and Teaching (NEET), published by Scineer Publishing PTE. LTD, is an open-access journal. NEET is devoted to publish education related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on education throughout the world. NEET welcomes all original manuscripts that do not be published in any other journals.Â

Â Â Â Â ISSN: 3029-200X(O)

Â Â Â Â Â Â Â Â Â 3029-1917(P)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Modern Economic Management Forum

Â

Modern Economic Management Forum (MEMF), published by Scineer Publishing PTE. LTD, is an open-access journal. MEMF is devoted to publish modern economic management related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on economic management throughout the world. MEMF welcomes all original manuscripts that do not be published in any other journals.Â

Â Â Â Â ISSN: 3029-2182(O)

Â Â Â Â Â Â Â Â Â 3029-2174(P)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Research and Commentary on Humanities and Arts

Research and Commentary on Humanities and ArtsÂ (RCHA), published by Scineer Publishing PTE. LTD, is an open-access journal. RCHA is devoted to publish humanities and arts related articles in the form of original papers, review papers and case studies. It also provides a platform for scholars to exchange the latest information in the related fields and aim to promote the research on humanities and arts throughout the world. RCHA welcomes all original manuscripts that do not be published in any other journals.Â

Â

Â Â Â

Â Â Â Â Â Â ISSN: 3029-2050(O)

Â Â Â Â Â Â Â Â Â Â Â 3029-2042(P)

Â Â Â Â Â Â Â

View Journal Current Issue Register

Scineer Publishing is a publisher of open access journals. Covering various disciplines and promoting interdisciplinary development, Scineer Publishing adheres to a principle of respecting academics, and publishes insightful and rigorous academic research contents.

12 Eu Tong Sen Street, #07-168, Singapore 059819

E-mail: scineerpub@126.com

Copyright © 2023 Scineer Publishing Pte. Ltd.

A Novel Algorithm to Improve the Performance of Mixture of Experts in Complex AI Tasks

Hanjie Xu*, Rui Li, Qiyu Chen

Abstract

We introduce Dynamic Layer Routing (DLR), a novel hierarchical sparse expert-mixing algorithm designed to improve performance
across complex AI tasks—including natural language processing, computer vision, and speech recognition. Building on the Mixture of Layer
Experts (MoLEx) framework, DLR dynamically routes inputs through selected network layers treated as “experts,” using a task-aware gating
mechanism that adapts to the difficulty and modality of each input. This layer-level routing promotes richer cross-layer and cross-modal information fusion while keeping additional parameter overhead minimal. Theoretical analysis demonstrates that DLR maintains comparable effective parameter budget to dense baselines yet achieves stronger robustness under distribution shifts. Empirical evaluations on GLUE benchmarks for NLP, CIFAR100 for vision, and LibriSpeech for ASR show consistent accuracy gains of 2.4–3.2% over MoLEx and traditional
sparse MoE models, with only marginal increases in computation. By enabling parallel expert processing with a lightweight shared-parameter
design, DLR offers an efficient and scalable approach for parameter-efficient fine-tuning in diverse multimodal and multi-task settings.

Keywords

Sparse mixtureofexperts; Transformer; Parameterefficient finetuning; Hierarchical Routing; Multimodal Learning

Full Text:

PDF

Included Database

References

[1] Jacob Devlin, MingWei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pretraining of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805, 2018.

[2] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan AllenZhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: LowRank

Adaptation of Large Language Models. arXiv preprint arXiv:2106.09685, 2021.

[3] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and

Sylvain Gelly. ParameterEfficient Transfer Learning for NLP. In ICML, pp. 2790–2799, 2019.

[4] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously Large Neural Networks: The SparselyGated MixtureofExperts Layer. In ICLR, 2017.

[5] William Fedus, Barret Zoph, and Noam Shazeer. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient

Sparsity. JMLR, 23(120):1–39, 2022.

[6] Aran Komatsuzaki, Joan Puigcerver, James LeeThorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani,

and Neil Houlsby. Sparse Upcycling: Training MixtureofExperts from Dense Checkpoints. arXiv preprint arXiv:2212.05055, 2022.

[7] Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and

Neil Houlsby. Scaling Vision with Sparse Mixture of Experts. NeurIPS, 34:8583–8595, 2021.

[8] Kenichi Kumatani, Robert Gmyr, Felipe Cruz Salinas, Linquan Liu, Wei Zuo, Devang Patel, Eric Sun, and Yu Shi. Building a Great

Multilingual Teacher with SparselyGated Mixture of Experts for Speech Recognition. arXiv preprint arXiv:2112.05820, 2021.

[9] Yonatan Belinkov and James Glass. Analysis Methods in Neural Language Processing: A Survey. TACL, 7:49–72, 2019.

[10] Tran P, Tropmann-Frick M. Global Contextualized Representations: Enhancing Machine Reading Comprehension with Graph Neural

Networks[M]//Information Modelling and Knowledge Bases XXXVI. IOS Press, 2025: 1-18..

[11] Wang H, Vaze S, Han K. Sptnet: An efficient alternative framework for generalized category discovery with spatial prompt tuning[J].

arXiv preprint arXiv:2403.13684, 2024.

[12] Wang Y, Liu Y, Zheng A, et al. Decoupled feature-based mixture of experts for multi-modal object re-identification[C]//Proceedings of

the AAAI Conference on Artificial Intelligence. 2025, 39(8): 8141-8149.

[13] Cai W, Jiang J, Wang F, et al. A survey on mixture of experts in large language models[J]. IEEE Transactions on Knowledge and Data

Engineering, 2025.

[14] Farina M, Ahmad U, Taha A, et al. Sparsity in transformers: A systematic literature review[J]. Neurocomputing, 2024: 127468.

[15] Nakamura T, Akiba T, Fujii K, et al. Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization[J]. arXiv preprint

arXiv:2502.19261, 2025.

DOI: http://dx.doi.org/10.70711/aitr.v2i10.7132

Refbacks

There are currently no refbacks.

Scineer Publishing is a publisher of open access journals. Covering various disciplines and promoting interdisciplinary development, Scineer Publishing adheres to a principle of respecting academics, and publishes insightful and rigorous academic research contents.

12 Eu Tong Sen Street, #07-168, Singapore 059819

E-mail: scineerpub@126.com

Copyright © 2023 Scineer Publishing Pte. Ltd.