How to debug Cpp codes in GGML development?

Build ggml from root directory


cmake -B build-cpu
cmake --build build-cpu --config Release -j 24

or with cuda backend


cmake -B build-cuda -DGGML_CUDA=ON
cmake --build build-cuda --config Release -j 24

Configurate debugging and running via tasks.json and launch.json under .vscode folder


// tasks.json
{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "build-with-ggml",
            "type": "shell",
            "command": "g++",
            "args": [
                "-g", "${file}",
                "-o", "${fileDirname}/out/${fileBasenameNoExtension}",
                "-I", "${workspaceFolder}/src",
                "-I", "${workspaceFolder}/include",
                "-L", "${workspaceFolder}/build/src",  // replace `build` with `build-cuda` if useing CUDA backend
                "-lggml",
                // "-DGGML_USE_CUDA"  // uncomment this line if useing CUDA backend
            ],
            "group": "build",
            "problemMatcher": [
                "$gcc"
            ],
            "dependsOn": "create-out-dir"
        },
        {
            "label": "create-out-dir",
            "type": "shell",
            "command": "mkdir",
            "args": [
                "-p",
                "${fileDirname}/out"
            ],
            "problemMatcher": []
        },
    ]
}

In tasks.json:

we configured a task named build-with-ggml, which will be used in the launch named C++ Debug with ggml via preLaunchTask
"-I", "${workspaceFolder}/src" and "-I", "${workspaceFolder}/include" are used to include necessary source files and header files of ggml library
"-L", "${workspaceFolder}/build/src" is used to indicate where the ggml shared library locates (which we have built in step1), and "-lggml" to actually link the ggml shared library (libggml.so in linux)


// launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "C++ Debug with ggml",
            "type": "cppdbg",
            "request": "launch",
            "program": "${fileDirname}/out/${fileBasenameNoExtension}",
            "args": [],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}",
            "environment": [
                {
                    "name": "LD_LIBRARY_PATH",
                    "value": "${workspaceFolder}/build/src:${env:LD_LIBRARY_PATH}" // replace `build` with `build-cuda` if useing CUDA backend
                }
            ],
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ],
            "preLaunchTask": "build-with-ggml",
            "miDebuggerPath": "/usr/bin/gdb",
            "logging": {
                // "trace": true,
                // "traceResponse": true,
                // "engineLogging": true
            }
        }
    ]
}

(Optional) Install gdb if haven't installed


sudo apt-get update
sudo apt-get install gdb

Use which gdb to check if gdb is successfully installed and the path matches with miDebuggerPath in launch.json

Debug / Run without debugging

VS Code extension: Graphviz Interactive Preview

With this extension, we don't have to bother converting .dot to .png for visualization

How to print a GGML tensor prettily?