Add lookup-table instructions (LUTI) Learning Path - #3663
Conversation
* Introduce the lookup-table instructions (LUTI): concepts, code implementation, and validation. Signed-off-by: Aude Vuilliomenet <aude.vuilliomenet@arm.com>
| </em> | ||
| </p> | ||
|
|
||
| This approach trades reconstruction accuracy for lower memory use. Its value also depends on decoding the packed codes efficiently. LUTI addresses that work by expanding low-bit codes directly into arithmetic-ready vector values. |
There was a problem hiding this comment.
"LUTI addresses that work by expanding low-bit codes directly into arithmetic-ready vector values."
Whilst grammatically correct, this sentence is prone to mis-parsing by the reader because "addresses" can be a noun or a verb and "work" can be a noun or intransitive verb. I think your intended meaning was:
noun(LUTI) verb(addresses) that noun(work) by...
However I initially read it as:
noun(LUTI addresses) that verb(work) by...
Consequently it read to me as an incomplete sentence, until after some time I was able to parse it the way you had intended. Let's make it easy on the reader by amending it to:
"LUTI achieves that by expanding low-bit codes directly into arithmetic-ready vector values."
Sorry for the long explanation!
There was a problem hiding this comment.
The second-from-right purple box should read "LUT [1]" or alternatively the second-from-right orange box should read "00".
|
|
||
| ## Identify LUTI responsibilities | ||
|
|
||
| Keep these boundaries in mind when using LUTI: |
There was a problem hiding this comment.
"Keep these boundaries in mind..."
->
"Keep these points in mind..."
| ```asm | ||
| smstart // Enable Streaming SVE mode and ZA/ZT0 state | ||
| zero {za} // Zero initialize accumulators for this output tile | ||
| ldr zt0, [x_lut] // load LUT into fixed 512-bit ZT0 table |
There was a problem hiding this comment.
Having used lut_i8_i2 on line 97, it would be good to reuse it here - or alternatively amend line 97 to use x_lut. This is just for consistency so the reader can tell it is the same register.
| ```asm | ||
| luti2 { z24.b - z27.b }, zt0, z16[0] // unpack 2-bit indices | ||
| luti2 { z4.b - z7.b }, zt0, z17[0] | ||
|
|
There was a problem hiding this comment.
Is this blank line intended? Consider deleting it, but it's fine if you prefer to keep it.
| {{< /tab >}} | ||
| {{< tab header="Linux host" language="bash">}} | ||
| wget https://dl.google.com/android/repository/android-ndk-r29-linux.zip | ||
| unzip android-ndk-r29-darwin.zip |
There was a problem hiding this comment.
unzip android-ndk-r29-darwin.zip
->
sudo apt update
sudo apt install unzip
unzip android-ndk-r29-linux.zip
| export ANDROID_NDK_HOME=$NDK_PATH | ||
| {{< /tab >}} | ||
| {{< tab header="Linux host" language="bash">}} | ||
| export NDK_PATH=$HOME/Android/android-ndk-r29/ |
There was a problem hiding this comment.
export NDK_PATH=$HOME/Android/android-ndk-r29/
->
export NDK_PATH=$HOME/android-ndk-r29/
| brew install android-platform-tools | ||
| {{< /tab >}} | ||
| {{< tab header="Linux host" language="bash">}} | ||
| sudo apt update |
There was a problem hiding this comment.
You can remove the "sudo apt update" on line 87 if you add it above in response to my comment on line 64.
| __[!REVIEW - ADD THE PUBLIC CODE-EXAMPLE ARCHIVE URL BEFORE PUBLISHING]__ | ||
|
|
||
| Download and extract the published code-example archive, then change to its `code` directory. | ||
|
|
There was a problem hiding this comment.
You can insert this code to download the files (remove the .clang-format line if you don't need it):
```bash
BASE_URL=https://raw.githubusercontent.com/ArmDeveloperEcosystem/arm-learning-paths/main/content/learning-paths/mobile-graphics-and-gaming/luti
mkdir code
cd code
for FILE in \
.clang-format \
Makefile \
example_1_luti_sme2.c \
luti_sme2_programming.c \
luti_sme2_programming_test.c; do
wget -q "$BASE_URL/code/$FILE" -O "$FILE"
done
```
|
|
||
| ## Download and explore the code examples | ||
|
|
||
| __[!REVIEW - ADD THE PUBLIC CODE-EXAMPLE ARCHIVE URL BEFORE PUBLISHING]__ |
There was a problem hiding this comment.
Remove this line and replace with the code in my comment below.
| From the same `code` directory, cross-compile for Android: | ||
|
|
||
| ```bash | ||
| make clean |
There was a problem hiding this comment.
First need: "sudo apt install build-essential"
|
|
||
| ```bash | ||
| make clean | ||
| make android |
There was a problem hiding this comment.
For me this gives:
error: Android NDK LLVM 21 or newer is required; found '' from /home/ubuntu/android-ndk-r29//toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android35-clang
make: *** [Makefile:97: check-android-llvm] Error 1
| adb push sme2_luti_android /data/local/tmp/sme2_luti_android | ||
| ``` | ||
|
|
||
| Start a new shell to access the device’s system from your development machine and runs the executable: |
|
|
||
| ## Choose an SME2 execution environment | ||
|
|
||
| You can run the examples using one of the following route: |
| ```text | ||
| rhs_packed[col] | ||
| bits [7:6] | [5:4] | [3:2] | [1:0] | ||
| RHS: [k3,col] | [k2,col] | [k1,col] | [0k,col] |
| From the `code` directory, compile `example_1_luti_sme2.c` with SME2 enabled: | ||
|
|
||
| ```bash | ||
| /opt/homebrew/opt/llvm/bin/clang \ |
There was a problem hiding this comment.
This is MacOS-specific. It would be good to also have alternate instructions for Linux + Android.
| `LUTI2` followed by four `SMOPA` instructions. | ||
|
|
||
| ```bash | ||
| /opt/homebrew/opt/llvm/bin/llvm-objdump -d example_1_luti_sme2 | \ |
There was a problem hiding this comment.
This is MacOS-specific. It would be good to also have alternate instructions for Linux + Android.
| - N = svcntb() = 512 / 8 = 64 | ||
| - K = 4 | ||
|
|
||
| | Block | Calculation | Size | |
There was a problem hiding this comment.
Consider adding a diagram so that readers can see this visually (though it's not essential).
| The examples cover the following combinations and have their base | ||
| in KleidiAI's matrix multiplication micro-kernels: | ||
|
|
||
| | Example | Decode | Arithmetic | Main concept | |
There was a problem hiding this comment.
Perhaps replace "One" and "Two" with something more descriptive
e.g:
"One" -> "arm_lp_gemm_luti4" or "FP16 LUTI4 + FMOPA"
"Two" -> "arm_lp_gemv_luti2_luti4" or "LUTI4 -> LUTI2 -> SDOT"
i.e. refer to the examples by their function name or their description in the output.
| | 3 | How many destination Z registers do you need the lookup to fill? | Choose x1, x2, or x4 to match the target operation. | | ||
| | 4 | How much of the source Z register fills the destination register group? | Source-register segment | | ||
|
|
||
| A *source segment* is the portion of one packed source Z register that fills the chosen destination group.</br> |
There was a problem hiding this comment.
It would be great to have a diagram to illustrate visually what a source segment is (though it's not essential).
| } | ||
| } | ||
| ``` | ||
| ## Two-stage LUTI4 and LUTI2 for GEMV using SDOT |
There was a problem hiding this comment.
## Example 2: Two-stage LUTI4 and LUTI2 for GEMV using SDOT
(to be consistent with line 60)
|
|
||
| ## Use Z-register tables with FEAT_LUT | ||
|
|
||
| `FEAT_LUT` expand the functionality of `FEAT_SME2` allowing to use scalable Z registers as the lookup-table source. This provide an alternative to using the `ZT0` register. |
There was a problem hiding this comment.
`FEAT_LUT` expands the functionality of `FEAT_SME2` allowing the use of scalable Z registers as the lookup-table source. This provides an alternative to using the `ZT0` register.
("expand" -> "expands", "to use" -> "the use of", "provide" -> "provides")
| one result Z register | ||
| ``` | ||
|
|
||
| Use this form for vector kernels that do not need `ZT0` register and where multiple LUT are |
| svuint8_t packed_indices = svld1_u8(pg8, rhs_indices); | ||
| rhs_indices += vl_b; | ||
| ``` | ||
| <del><code> svldr_zt(0, zt0_luti4);</code></del> |
There was a problem hiding this comment.
It is unclear what the strikethrough means. Are you trying to say "Replace svldr_zt(0, zt0_luti4) with:"? If so, it might be clearer to say that explicitly.
| svluti4_lane_u8(luti4_table_z, packed_indices, /* segment */ 0); | ||
| .. | ||
| ``` | ||
| <del><code> svldr_zt(0, zt0_luti2);</code></del> |
There was a problem hiding this comment.
It is unclear what the strikethrough means. Are you trying to say "Replace svldr_zt(0, zt0_luti2) with:"? If so, it might be clearer to say that explicitly.
2 participants
Introduce the LUTI concepts, code implementation, and validation.
Before submitting a pull request for a new Learning Path, please review Create a Learning Path
Please do not include any confidential information in your contribution. This includes confidential microarchitecture details and unannounced product information.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the Creative Commons Attribution 4.0 International License.