Panduan Teknikal: Compile llama.cpp di Debian 12/13 dan Cross Compile ARM64

1. Pengenalan llama.cpp ialah runtime inference LLM berasaskan C/C++ yang popular kerana ringan, pantas, dan sesuai untuk menjalankan model GGUF secara local. Ia boleh digunakan pada: Server x86_64 Dalam deployment sebenar, terdapat dua pendekatan utama: Native build Bahagian 1 — Persediaan Debian 12/13 1.1 Install dependency asas sudo apt update sudo apt install -y \ git \ build-essential \ cmake \ ninja-build \ pkg-config Komponen utama: Package Fungsi Bahagian 2 — Clone llama.cpp git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp Semak versi: git log -1 --oneline Bahagian 3 — Compile Native (Mesin Sama) Contoh: Debian 12/13 x86_64 3.1 Configure CMake Build menggunakan Ninja: cmake -B build \ -G Ninja \ -DCMAKE_BUILD_TYPE=Release 3.2 Compile ninja -C build -j$(nproc) atau: cmake --build build plaintext 3.3 Hasil build Semak: ls build/bin plaintext llama-cli llama-server llama-bench llama-perplexity shell Bahagian 4 — Enable OpenBLAS (Pilihan) OpenBLAS boleh membantu operasi matrix CPU. Install: sudo apt install libopenblas-dev cmake cmake -B build \ -G Ninja \ -DCMAKE_BUILD_TYPE=Release \ -DGGML_BLAS=ON \ -DGGML_BLAS_VENDOR=OpenBLAS shell ninja -C build Nota Penting: CMake Cache Jika pernah configure dengan: -DGGML_BLAS=ON kemudian buang option tersebut, CMake masih menyimpan konfigurasi lama. Contoh masalah: BLAS not found Penyelesaian: rm -rf build Kemudian configure semula. Sentiasa ingat: CMakeCache.txt menyimpan konfigurasi lama. Bahagian 5 — Cross Compile x86_64 → ARM64 Contoh: PC Debian 12 x86_64 Kelebihan: Compile lebih cepat 5.1 Install ARM64 cross compiler sudo apt install -y \ gcc-12-aarch64-linux--gnu\ g++-12-aarch64-linux-gnu sudo apt install -y \ gcc-13-aarch64-linux--gnu\ g++-13-aarch64-linux-gnu Semak: aarch64-linux-gnu-gcc --version 5.2 Configure cross build Bersihkan dahulu: rm -rf build-arm Kemudian: cmake -B build-arm \ -G Ninja \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_SYSTEM_NAME=Linux \ -DCMAKE_SYSTEM_PROCESSOR=aarch64 \ -DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc \ -DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ 5.3 Compile ninja -C build-arm -j$(nproc) Hasil: ls build-arm/bin Bahagian 6 — Semak Architecture Binary Gunakan: file build-arm/bin/llama-server Contoh output berjaya: ELF 64-bit LSB pie executable, ARM aarch64, dynamically linked Maksud: Output Maksud Jika compile ARM64 tetapi check pada PC x86: ldd llama-server boleh gagal: not a dynamic executable Sebab: PC: x86_64 loader Binary: ARM64 loader Contoh: Shared library: [libllama.so] Shared library: [libggml.so] Shared library: [libstdc++.so.6] Cari semua .so find build-arm -name "*.so" Contoh: libllama.so libggml.so libggml-base.so libggml-cpu.so Semak architecture: file build-arm/bin/*.so Output: ARM aarch64 Bahagian 8 — Dynamic vs Static Binary Semak: file llama-server Contoh dynamic: dynamically linked Perlu: lib*.so Contoh static: statically linked Tidak perlu .so. Bahagian 9 — Installation ke Linux Binary: /usr/local/bin Library: /usr/local/lib Contoh: sudo cp llama-server /usr/local/bin/ sudo cp llama-cli /usr/local/bin/ sudo cp *.so /usr/local/lib/ sudo ldconfig Pilihan appliance / embedded Untuk SBC: /opt/llama.cpp/ llama-server llama-cli libllama.so libggml.so Kemudian: export LD_LIBRARY_PATH=/opt/llama.cpp Sesuai untuk: Orange Pi Bahagian 10 — Deploy ke Orange Pi Copy: scp build-arm/bin/llama-server \ orangepi:/usr/local/bin/ scp build-arm/bin/llama-cli \ orangepi:/usr/local/bin/ Jika perlu: scp build-arm/bin/*.so \ orangepi:/usr/local/lib/ Pada Orange Pi: sudo ldconfig Semak: uname -m Expected: aarch64 Bahagian 11 — Cadangan Production Architecture Untuk sistem AI agent: +----------------+ Kelebihan: Go agent tidak perlu embed model Workflow yang stabil: Native ninja -C build rm -rf build-arm cmake -B build-arm \ ninja -C build-arm aarch64-linux-gnu-readelf -d llama-server | grep NEEDED find . -name "*.so" Dengan proses ini, satu mesin Debian 12/13 boleh menjadi build server untuk menghasilkan node AI ARM64 seperti Orange Pi, Raspberry Pi, atau edge inference appliance.