Limits

This page summarizes current Pinecone limits and known limitations.

General limitations

• Upserts

  • Max size for an upsert request is 2MB. Recommended upsert limit is 100 vectors per request.

  • Pinecone is eventually consistent, so there can be a slight delay before upserted records are available to query.

    After upserting records, use the describe_index_stats operation to check if the current vector count matches the number of records you expect, although this method may not work for pod-based indexes with multiple replicas.

  • Dense vectors must contain at least one non-zero value.

  • Max dimensionality for dense vectors is 20,000.

  • Sparse vectors can contain no more than 1000 non-zero values.

  • Max dimensionality for sparse vectors is 4.2 billion.

  • Records with sparse vector values must also contain dense vector values.

  • Only indexes using the dotproduct distance metric support sparse-dense vectors.

    Upserting sparse-dense vectors into indexes with a different distance metric will succeed, but querying will return an error.

  • Indexes created before February 22, 2023 do not support sparse vectors.

• Metadata
  • Max metadata size per vector is 40 KB.
  • Null metadata values are not supported. Instead of setting a key to hold a null value, we recommend you remove that key from the metadata payload.
• Queries
  • Max value for top_k, the number of results to return, is 10,000.
• Fetches and deletes
  • Max vectors per fetch or delete request is 1,000.

Serverless index limitations

Serverless indexes are in public preview and are available only on AWS in the us-west-2 and us-east-1 regions. Check the current limitations and test thoroughly before using it in production.

• Each project is limited to 20 serverless indexes.
• Each serverless index is limited to 10,000 namespaces.
• Each serverless namespace is limited to 1B vectors.
• During the public preview period, requests are limited as follows:
  • query requests are limited to 1000 read units/second.
  • upsert, delete, and update requests, combined, are limited to 50,000 write units/second.
  • list, fetch, and describe_index_stats requests, combined, are limited to 100 requests/second.
• Serverless indexes do not support the following features:
  • Collections
  • Deleting records by metadata
    Instead, you can delete records by ID prefix.
  • Selective metadata indexing
  • Configuring indexes (configure_index())
    Because serverless indexes scale automantically based on usage, this operation is not relevant
  • Describing indexes with metadata filtering

Pod-based index limitations

• Pod storage capacity
  • Each p1 pod has enough capacity for 1M vectors with 768 dimensions.
  • Each s1 pod has enough capacity for 5M vectors with 768 dimensions.
• Metadata
  • Metadata with high cardinality, such as a unique value for every vector in a large index, uses more memory than expected and can cause the pods to become full.
• Collections
  • You cannot query or write to a collection after its creation. For this reason, a collection only incurs storage costs.
  • You can only perform operations on collections in the current Pinecone project.
• Sparse-dense vectors
  • Only s1 and p1 pod-based indexes using the dotproduct distance metric support sparse-dense vectors.

Starter index limitations

• Organizations on the Starter plan are limited to one project containing one free pod-based starter index.

  If you need more than a single starter index, upgrade to the Standard or Enterprise plans.

• Starter indexes are limited to 100,000 records.

• Starter indexes are limited to 100 namespaces.

  If you need more than 100 namespaces, you may wish to use metadata filtering to support multitenancy in your project. Another alternative is to upgrade to the Standard or Enterprise plans and use index types that support more namespaces.

• Starter indexes can be hosted only in the gcp-starter environment, which is us-central-1 (Iowa) region of the GCP cloud.

• Starter indexes do not support the following features:

  • Replicas

  • Collections

    Collections may not be necessary or appropriate for starter indexes. Collections serve two primary purposes: decreasing usage by archiving inactive indexes, and experimenting with different index configurations. However, starter indexes neither incur usage costs nor specify pod types or sizes.

  • Deleting records by metadata

    In some cases, you may be able to delete records by ID instead.

  • Selective metadata indexing

  • Configuring indexes (configure_index())

    Because you can have only one starter index, this operation is not supported.

  • Describing indexes with metadata filtering