Enhanced Syndrome-based Reliability Decoding for Error Correction Code Transformers
- Alternative Title
- 오류 정정 코드 변환기를 위한 향상된 신드롬 기반 신뢰성 디코딩
- Abstract
- In this work, dynamically adaptive refinement masking (DARM) and mirror- sharing variational U-shaped architecture are proposed in order to improve the performance of error correction code transformer (ECCT). Instead of fixed binary masking, DARM module is designed to create unique masking for each attention head that reinforces the self-attention mechanism by further enhanc-ement of the contrast in the attention map with dynamic magnitudes taking the distribution of the attention weight as reference. Furthermore, under the use of sequential neural architecture, DARM modules are designed to be sequentially connected, creating an iterative refinement effect. For moderate coding lengths, the mirror-sharing variational U-shaped architecture is introduced to enhance the overall efficiency of the transformer-based decoder. The U-shaped architecture with variational-autoencoder-like skip-connection provides a segmentation like behavior that operates well with moderate length codes, especially at low coding rates. As the U-shaped model requires a certain level of depth to achieve desirable performance, an architectural-level parameter-sharing scheme called mirror-sharing is introduced to effectively scale the U-shaped model to achieve better efficiency and performance. Experimental results show considerable improvements in bit error rates compared to the baseline ECCT, while also significantly increasing the training convergence speed.
- Author(s)
- 웬 당 트락
- Issued Date
- 2024
- Awarded Date
- 2024-08
- Type
- Dissertation
- Keyword
- Error correction codes; Application of Deep learning in communication; study of transformers neural network model
- URI
- https://oak.ulsan.ac.kr/handle/2021.oak/13325
http://ulsan.dcollection.net/common/orgView/200000805509
- Authorize & License
-
- Files in This Item:
-
Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.