Abstract
Transformer models such as BERT and Vision Transformer~(ViT) achieve strong performance via densely parameterized attention backbones. However, the least significant bits~(LSBs) of their 32-bit floating-point weights can be abused as covert channels to conceal malicious payloads, posing a serious threat to the AI model supply chain. We propose \GS (\GSabbr), a lightweight, post-training, zero-data sanitization method that completely replaces the declared mantissa-LSB channel with a Gray-code-guided low-transition sequence. Complete payload-independent overwrite, whether keyed or public, makes the sanitized target bits independent of the embedded payload and gives that declared channel zero capacity. Gray coding supplies overwrite structure, while a keyed per-tensor phase supplies pattern diversity. Benchmarked against seven post-training defenses on four Transformer model presets and two real-world malware payloads, \GSabbr maintains sub-$1\%$ accuracy impact and achieves $49.96\pm0.66$ percentage-point Recovery Reduction (RR) under five implemented attacker variants. Because pre-defense recovery is effectively $100\%$, RR near 50 percentage points corresponds to post-sanitization bit accuracy at binary chance. Its main empirical advantage is stable near-chance sanitization with substantially smaller weight-distribution shift than the evaluated near-chance baselines PatternMask (PM) and Post-Training Quantization (PTQ).
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Foundjem, A., Chuang, T. H., Khomh, F., & Merzouk, M. A. (2026). GrayShield: Bit-Level Sanitization for Transformer Model Supply-Chain Security. https://omanscience.com/en/articles/grayshield-bit-level-sanitization-for-transformer-model-supply-chain-security
MLA 9
Foundjem, Armstrong, et al. "GrayShield: Bit-Level Sanitization for Transformer Model Supply-Chain Security." https://omanscience.com/en/articles/grayshield-bit-level-sanitization-for-transformer-model-supply-chain-security.
Chicago (author–date)
Foundjem, Armstrong, Tsung-Hsien Chuang, Foutse Khomh, and Mohamed Amine Merzouk. 2026. "GrayShield: Bit-Level Sanitization for Transformer Model Supply-Chain Security." https://omanscience.com/en/articles/grayshield-bit-level-sanitization-for-transformer-model-supply-chain-security.
Harvard
Foundjem, A., Chuang, T. H., Khomh, F. and Merzouk, M. A. (2026) 'GrayShield: Bit-Level Sanitization for Transformer Model Supply-Chain Security', Available at: https://omanscience.com/en/articles/grayshield-bit-level-sanitization-for-transformer-model-supply-chain-security.
Vancouver
Foundjem A, Chuang TH, Khomh F, Merzouk MA. GrayShield: Bit-Level Sanitization for Transformer Model Supply-Chain Security. https://omanscience.com/en/articles/grayshield-bit-level-sanitization-for-transformer-model-supply-chain-security
IEEE
A. Foundjem, T. H. Chuang, F. Khomh, and M. A. Merzouk, "GrayShield: Bit-Level Sanitization for Transformer Model Supply-Chain Security," https://omanscience.com/en/articles/grayshield-bit-level-sanitization-for-transformer-model-supply-chain-security.