KLI

Enhanced Syndrome-based Reliability Decoding for Error Correction Code Transformers

Metadata Downloads
Alternative Title
오류 정정 코드 변환기를 위한 향상된 신드롬 기반 신뢰성 디코딩
Abstract
In this work, dynamically adaptive refinement masking (DARM) and mirror- sharing variational U-shaped architecture are proposed in order to improve the performance of error correction code transformer (ECCT). Instead of fixed binary masking, DARM module is designed to create unique masking for each attention head that reinforces the self-attention mechanism by further enhanc-ement of the contrast in the attention map with dynamic magnitudes taking the distribution of the attention weight as reference. Furthermore, under the use of sequential neural architecture, DARM modules are designed to be sequentially connected, creating an iterative refinement effect. For moderate coding lengths, the mirror-sharing variational U-shaped architecture is introduced to enhance the overall efficiency of the transformer-based decoder. The U-shaped architecture with variational-autoencoder-like skip-connection provides a segmentation like behavior that operates well with moderate length codes, especially at low coding rates. As the U-shaped model requires a certain level of depth to achieve desirable performance, an architectural-level parameter-sharing scheme called mirror-sharing is introduced to effectively scale the U-shaped model to achieve better efficiency and performance. Experimental results show considerable improvements in bit error rates compared to the baseline ECCT, while also significantly increasing the training convergence speed.
Author(s)
웬 당 트락
Issued Date
2024
Awarded Date
2024-08
Type
Dissertation
Keyword
Error correction codesApplication of Deep learning in communicationstudy of transformers neural network model
URI
https://oak.ulsan.ac.kr/handle/2021.oak/13325
http://ulsan.dcollection.net/common/orgView/200000805509
Alternative Author(s)
Nguyen Dang Trac
Affiliation
울산대학교
Department
일반대학원 전기전자컴퓨터공학과
Advisor
Sunghwan Kim
Degree
Master
Publisher
울산대학교 일반대학원 전기전자컴퓨터공학과
Language
eng
Rights
울산대학교 논문은 저작권에 의해 보호받습니다.
Appears in Collections:
Computer Engineering & Information Technology > 1. Theses(Master)
Authorize & License
  • Authorize공개
Files in This Item:

Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.