icl-four-primitives-each-implementable-in-single-layer

IN premise — summaries/2026/08/24/akyurek-2023-icl-algorithm-s0-abstract-chunk-1.md

Created 2026-08-24T17:10:49+00:00

In Akyurek et al. (2023), the four computational primitives—mov (copy), mul (matrix multiply within a column), div (element-wise divide), and aff (affine combination)—are each implementable by a single transformer decoder layer, serving as building blocks for all algorithm constructions.

Summary

Each of the four basic computational steps used to build algorithms in transformers (copying data, multiplying within a group, dividing element-wise, and blending with a shift) can be handled by one decoder layer on its own. This matters because it means complex multi-step algorithms map cleanly onto a stack of layers, giving a concrete lower bound on how many layers a given computation requires.