Power Load Forecasting Based on TCN and Lightweight Autoformer

doi:10.3969/j.issn.1006-2475.2025.04.002

Abstract

Abstract: The accuracy of power load forecasting is crucial for energy conservation and emission reduction， and higher accuracy can enable power companies to make more reasonable plans and improve economic benefits. Although Autoformer， based on the improved Transformer architecture， has achieved good results in sequence prediction tasks， it did not fully consider the causal relationship of time when extracting temporal features， and there is too much redundant information in the attention layer， which leads to a decrease in model accuracy and memory consumption. To address these issues， this paper proposes a power load forecasting method that combines Time Convolutional Network （TCN） and an improved lightweight Autoformer model. Firstly， a time convolutional network is introduced into the Autoformer encoder to provide a larger receptive field and fully consider the causal relationship of the samples. Then， a distillation mechanism is added between the autocorrelation attention layers to reduce the number of model parameters. Finally， the results of experiment on five public datasets showed that the lightweight Autoformer combined with TCN reduced MSE and MAE by 8.95% to 32.40% and 4.91% to 15.51% respectively compared to the original model， and the prediction performance is significantly better than the other four mainstream methods， demonstrating its excellent performance.

Key words: Transformer, Autoformer, temporal convolutional network, attention distilling, load forecasting

CLC Number:

TP391

LI Ming, SHI Chaoshan, WEN Guihao, LUO Yonghang, TAN Yunfei. Power Load Forecasting Based on TCN and Lightweight Autoformer[J]. Computer and Modernization, 2025, 0(04): 6-11.

References

［1］董红斌，韩爽，付强. 基于AR与DNN联合模型的地理传感器时间序列预测［J］. 计算机科学， 2023，50（11）：41-48.
［2］苏佳，李高雅，张新生. 融合时空特征的GCN-LSTM西北地区沙尘天气预测模型研究［J］. 干旱区资源与环境， 2024，38（5）：111-120.
［3］孙子雨，任燃，魏曦哲. 基于DTW-TCN的股票分类及预测研究［J］. 计算机与现代化， 2023（8）：31-37.
［4］ GE M， ZHANG J F， WU J F， et al. ARIMA-FSVR hybrid method for high-speed railway passenger traffic forecasting［J］. Mathematical Problems in Engineering， 2021，2021（1）. DOI： 10.1155/2021/9961324.
［5］ WAN A P， CHANG Q， AL-BUKHAITI K， et al. Short-term power load forecasting for combined heat and power using CNN-LSTM enhanced by attention mechanism［J］. Energy， 2023，282. DOI： 10.10161/j.energy.2023.128274.
［6］ DUDEK G， PEŁKA P. Pattern similarity-based machine learning methods for mid-term load forecasting： A comparative study［J］. Applied Soft Computing， 2021，104. DOI：
10.1016/j.asoc.2021.107223.
［7］ MI J W， FAN L B， DUAN X C， et al. Short-term power load forecasting method based on improved exponential smoothing grey model［J］. Mathematical Problems in Engineering， 2018，2018（1）. DOI: 10.1155/2018/3894723.
［8］ PEI X Y. Prophet algorithm-based power load forecasting model［C］// 2023 IEEE 3rd International Conference on Electronic Technology， Communication and Information （ICETCI）. IEEE， 2023：1498-1503.
［9］ CHODAKOWSKA E， NAZARKO J， NAZARKO Ł. ARIMA models in electrical load forecasting and their robustness to noise［J］. Energies， 2021，14（23）. DOI： 10.3390/en1423
7952.
［10］ ZULFIQAR M， KAMRAN M， RASHEED M B， et al. Hyperparameter optimization of support vector machine using adaptive differential evolution for electricity load forecasting［J］. Energy Reports， 2022，8：13333-13352.
［11］ LARA-BENÍTEZ P， CARRANZA-GARCÍA M， RIQUELM
E J C. An experimental review on deep learning architectures for time series forecasting［J］. International Journal of Neural Systems， 2021，31（3）. DOI: 10.1142/S0129065721
300011.
［12］ JALALI S M J， AHMADIAN S， KHOSRAVI A， et al. A novel evolutionary-based deep convolutional neural network model for intelligent load forecasting［J］. IEEE Transactions on Industrial Informatics， 2021，17（12）：8243-8253.
［13］ KONG W C， DONG Z Y， JIA Y W， et al. Short-term residential load forecasting based on LSTM recurrent neural network［J］. IEEE Transactions on Smart Grid， 2019，10（1）： 841-851.
［14］ HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997，9（8）：1735-1780.
［15］张淑娴，江文韬，陈玉花，等. 基于二次模态分解的LSTM短期电力负荷预测［J］.科学技术与工程， 2024，24（7）：2759-2766.
［16］尹春杰，肖发达，李鹏飞，等. 基于LSTM神经网络的区域微网短期负荷预测［J］. 计算机与现代化， 2022（4）：7-11.
［17］崔星，李晋国，张照贝，等. 基于改进粒子群算法优化LSTM的短期电力负荷预测［J］.电测与仪表，2024，61（1）：131-136.
［18］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. ACM， 2017：6000-6010.
［19］ LIN T Y， WANG Y X， LIU X Y， et al. A survey of transformers［J］. AI Open， 2022，3：111-132.
［20］ LI S Y， JIN X Y， XUAN Y， et al. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting［C］// Proceedings of the 33rd International Conference on Neural Information Processing Systems. ACM， 2019， 471：5243-5253.
［21］ KITAEV N， KAISER Ł， LEVSKAYA A. Reformer： The efficient transformer［J］. arXiv preprint arXiv：2001.04451，
2020.
［22］ WU H X， XU J H， WANG J M， et al. Autoformer： Decomposition transformers with auto-correlation for long-term series forecasting［C］// Proceedings of the 35th International Conference on Neural Information Processing Systems. ACM， 2021：22419-22430.
［23］ BAI S J， KOLTER J Z， KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling［J］. arXiv preprint arXiv：1803.01271， 2018.
［24］ CLEVERT D A， UNTERTHINER T， HOCHREITER S. Fast and accurate deep network learning by exponential linear units （ELUs）［J］. arXiv preprint arXiv：1511.07289， 2015.
［25］ ZHANG X， YOU J L. A gated dilated causal convolution based encoder-decoder for network traffic forecasting［J］. IEEE Access， 2020，8：6087-6097.
［26］ ZHANG L， NA J M， ZHU J， et al. Spatiotemporal causal convolutional network for forecasting hourly PM2. 5 concentrations in Beijing， China［J］. Computers & Geosciences， 2021，155. DOI： 10.1016/j.cageo.2021.104869.
［27］ WOO G， LIU C H， SAHOO D， et al. ETSformer： Exponential smoothing transformers for time-series forecasting［J］. arXiv preprint arXiv：2202.01381，2022.
［28］ LIU M H， ZENG A L， CHEN M X， et al. SCINet： Time series modeling and forecasting with sample convolution and interaction［C］// Proceedings of the 36th International Conference on Neural Information Processing Systems. ACM， 2022，35：5816-5828.

[1]	HOU Congying, YANG Wengqing, WANG Zhao, CHENG Cong. Speech Enhancement Based on Time-frequency Self-attention Residual Temporal#br# Convolutional Networks [J]. Computer and Modernization, 2024, 0(09): 20-24.
[2]	SUN Zi-yu, REN Ran, WEI Xi-zhe. Research on Stock Classification and Forecast Based on DTW-TCN [J]. Computer and Modernization, 2023, 0(08): 31-37.