You can, you just don't query by the embedding field as well, only add the embeddings inside the vector query