Hello,

I previously submitted a not-so-smart issue. Thank you everyone for your answers. I have come to understand the index limitations of PG, as well as some other useful knowledge. Particularly, pgvector: Fewer dimensions are better. Based on this article, I decided to tackle the dimensionality issue of ollama from a different angle.

During this period, I have written several dimensionality reduction algorithms. From the perspective of effectiveness and implementation complexity, the simplest method of partitioning and taking the average value is a good choice.

This function is not complicated; on the contrary, among the algorithms I have tried so far, this method is the simplest:

static inline float sum_float(const float *matrix, int start, int stop) {
    float result = 0.0;
    for (int i = start; i < stop; i++) {
        result += matrix[i];
    }
    return result;
}

/*
 * reduce vector dims by norm in fixed ranges
 */
PGDLLEXPORT PG_FUNCTION_INFO_V1(vector_norm_reduce);

Datum
vector_norm_reduce(PG_FUNCTION_ARGS) {
    Vector *vec = PG_GETARG_VECTOR_P(0);
    int32 reduce_to = PG_GETARG_INT32(1);
    Vector *result;
    int dim = vec->dim;
    float *values = vec->x;

    if (reduce_to < 2)
        ereport(ERROR,
                (errcode(ERRCODE_DATA_EXCEPTION),
                        errmsg("vector cannot reduce to a vector less than 2 by shear")));

    if (reduce_to >= dim)
        ereport(ERROR,
                (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
                        errmsg("vector cannot reduce to more than %d dimensions", dim)));

    CheckDim(dim);
    CheckDim(reduce_to);

    result = InitVector(reduce_to);

    int step = vec->dim / reduce_to;
    for (int idx = 0; idx < reduce_to; idx++) {
        int start = idx * step;
        int stop = (idx + 1) * step - 1;
        if (stop > reduce_to - 1) {
            stop = reduce_to - 1;
        }
        float range = stop - start;
        result->x[idx] = sum_float(vec->x, start, stop) / range;
    }

    PG_RETURN_POINTER(result);
}

I have collected hundreds of sentences of text, including both Chinese and English, some technical articles and literary works, and compared the 4096-dimensional embedding vectors generated directly from ollama with the results of different compression methods. The norm algorithm compressed to 512 dimensions achieved the best results, and most of the time its search results are closer to the original 4096-dimensional vectors.

The Python script test/python/generate_and_save.py is used to generate test data, for example:

python test/python/generate_and_save.py test/python/stories.txt

Another Python script test/python/ask.py is used to compare search results, executed like this:

python test/python/ask.py "They're playing our song."

Perhaps, in addition to me, there may be users who need to generate compressed vectors of lower dimensions from high-dimensional embedding vectors. I hope this function can be of help.