Queries¶
Querying is where Saffier's Django-like API meets SQLAlchemy Core's execution model.
Managers hand out querysets, querysets accumulate query state lazily, and the
database is only touched when you execute a terminal operation such as get(),
all(), create(), update(), or delete().
That means a good mental model for Saffier queries is:
- build a queryset
- keep chaining until the intent is complete
- execute once
If you have not yet read the models and managers pages, read those first. Query behavior makes more sense once you know where the queryset comes from.
A realistic querying example¶
recent_active_users = (
User.query
.filter(is_active=True, email__icontains="@example.com")
.select_related("profile")
.order_by("-created_at")
.limit(50)
)
rows = await recent_active_users.all()
This example shows the most important queryset traits:
- every method returns a new queryset
- nothing executes until
all()runs - relation loading is part of the query plan, not a side effect afterwards
QuerySet¶
When making queries within Saffier, this return or an object if you want only one result or a
queryset which is the internal representation of the results.
If you are familiar with Django querysets, this is almost the same and by almost is because saffier restricts loosely queryset variable assignments.
Let us get familar with queries.
Let us assume you have the following User model defined.
import saffier
from saffier import Database, Registry
database = Database("sqlite:///db.sqlite")
models = Registry(database=database)
class User(saffier.Model):
is_active = saffier.BooleanField(default=True)
first_name = saffier.CharField(max_length=50)
last_name = saffier.CharField(max_length=50)
email = saffier.EmailField(max_lengh=100)
password = saffier.CharField(max_length=1000)
class Meta:
registry = models
class User(saffier.Model):
user = saffier.ForeignKey(User, on_delete=saffier.CASCADE)
class Meta:
registry = models
As mentioned before, Saffier returns querysets and simple objects and when querysets are returned
those can be chained together, for example, with filter() or limit().
await User.query.filter(is_active=True).filter(first_name__icontains="a").order_by("id")
Do we really need two filters here instead of one containing both conditions? No, we do not but this is for example purposes.
Internally when querying the model and with returning querysets, Saffier runs the all().
This can be done manually by you or automatically by the ORM.
Let us refactor the previous queryset and apply the manual all().
await User.query.filter(is_active=True, first_name__icontains="a").order_by("id").all()
And that is it. Of course there are more filters and operations that you can do with the ORM and we will be covering that in this document but in a nutshell, querying the database is this simple.
Load the foreign keys beforehand with select related¶
Select related is a functionality that follows the foreign-key relationships by selecting any
additional related object when a query is executed. You can imagine it as a classic join.
The difference is that when you execute the select_related, the foreign keys of the model being used by that operation will be opulated with the database results.
You can use the classic select_related:
await Profile.query.select_related("user").get(id=1)
Or you can use the load() function of the model for the foreign key. Let us refactor the example above.
profile = await Profile.query.get(id=1)
await profile.user.load()
The load() works on any foreign key declared and it will automatically load the data into that
field.
Returning querysets¶
There are many operations you can do with the querysets and then you can also leverage those for your use cases.
Exclude¶
The exclude() is used when you want to filter results by excluding instances.
users = await User.query.exclude(is_active=False)
Filter¶
Django-style¶
These filters are the same Django-style lookups.
users = await User.query.filter(is_active=True, email__icontains="gmail")
The same special operators are also automatically added on every column.
- in - SQL
INoperator. - exact - Filter instances matching the exact value.
- iexact - Filter instances mathing the exact value but case-insensitive.
- contains - Filter instances that contains a specific value.
- icontains - Filter instances that contains a specific value but case-insensitive.
- lt - Filter instances having values
Less Than. - lte - Filter instances having values
Less Than Equal. - gt - Filter instances having values
Greater Than. - gte - Filter instances having values
Greater Than Equal. - isempty - Filter instances where a field holds its Saffier-defined empty value.
- isnull - Filter instances where a column is
NULLor notNULL.
Example¶
users = await User.query.filter(email__icontains="foo")
users = await User.query.filter(id__in=[1, 2, 3])
users = await User.query.filter(name__isempty=True)
users = await User.query.filter(last_login__isnull=True)
SQLAlchemy style¶
Since Saffier builds on SQLAlchemy expressions, it is also possible to do queries in SQLAlchemy style. The filter accepts also those.
If you need direct class-attribute access such as User.id instead of
User.columns.id, see SQLAlchemy compatibility mode.
Example¶
users = await User.query.filter(User.columns.email.contains("foo"))
users = await User.query.filter(User.columns.id.in_([1, 2, 3]))
Q expressions¶
For nested boolean predicates, use Q.
from saffier import Q
users = await User.query.filter((Q(name="Adam") & Q(email__icontains="saffier")) | ~Q(id=1))
Local OR¶
Use local_or() to combine OR clauses with existing queryset filters:
users = await User.query.filter(is_active=True).local_or(email__icontains="example.com")
Warning
The columns refers to the columns of the underlying SQLAlchemy table.
Limit¶
Limiting the number of results. The LIMIT in SQL.
users = await User.query.limit(1)
users = await User.query.filter(email__icontains="foo").limit(2)
Offset¶
Applies the office to the query results.
users = await User.query.offset(1)
users = await User.query.filter(is_active=False).offset(2)
Since you can chain the querysets from other querysets, you can aggregate multiple operators in one go as well.
await User.query.filter(email__icontains="foo").limit(5).order_by("id")
Batch size¶
When iterating asynchronously, you can set a chunk size for database reads:
async for user in User.query.order_by("id").batch_size(100):
...
Extra and reference selects¶
extra_select() adds SQLAlchemy expressions to the SELECT list.
reference_select() maps already-selected values back onto model attributes, including nested
related objects.
import sqlalchemy
queryset = User.query.extra_select(sqlalchemy.literal(1).label("marker"))
queryset = queryset.reference_select({"score": "marker"})
Reference paths can target related models:
queryset = Profile.query.select_related("user").reference_select(
{"user": {"profile_name": "name"}, "user_name": "user__name"}
)
If you need the raw SQLAlchemy statement for subqueries or aggregates, use as_select():
user_select = await User.query.filter(is_active=True).as_select()
total = sqlalchemy.func.count().select().select_from(user_select.subquery())
Order by¶
Classic SQL operation and you need to order results.
Order by descending id and ascending email
users = await User.query.order_by("email", "-id")
Order by ascending id and ascending email
users = await User.query.order_by("email", "id")
Lookup¶
This is a broader way of searching for a given term. This can be quite an expensive operation so be careful when using it.
users = await User.query.lookup(term="gmail")
Distinct¶
Applies SQL DISTINCT semantics to a queryset.
users = await User.query.distinct()
users = await User.query.distinct("email")
Use distinct(False) to clear a previously applied distinct clause on a cloned queryset.
Warning
Not all SQL databases support DISTINCT ON fields equally. PostgreSQL does, but MySQL and
SQLite have limitations here.
Be careful to know and understand where this should be applied.
Set operations¶
Saffier supports SQL set operations between querysets of the same model.
| Operation | Description | SQL Equivalent |
|---|---|---|
.union(qs2) |
Combines both querysets, removing duplicates. | UNION |
.union_all(qs2) |
Combines both querysets, keeping duplicates. | UNION ALL |
.intersect(qs2) |
Returns only rows appearing in both querysets. | INTERSECT |
.intersect_all(qs2) |
Uses the ALL variant when the backend supports it. |
INTERSECT ALL |
.except_(qs2) |
Returns rows from the first queryset that are not in the second. | EXCEPT |
.except_all(qs2) |
Uses the ALL variant when the backend supports it. |
EXCEPT ALL |
All set operations return a combined queryset, so outer queryset modifiers still apply to the merged result:
combined = User.query.filter(is_active=True).union(
User.query.filter(is_staff=True)
)
rows = await combined.order_by("email").offset(5).limit(10)
The outer queryset supports the same result helpers you would use on a regular queryset:
await combined.values(["id", "email"])
await combined.exists()
await combined.count()
await combined.first()
await combined.last()
The duplicate-preserving variants are also available directly:
User.query.union_all(other_queryset)
User.query.intersect_all(other_queryset)
User.query.except_all(other_queryset)
Warning
Set operations require both querysets to use the same model, the same database connection,
and the same selected column shape. If one side uses only() or defer(), the other side
must project the same columns.
Deferred and reduced projections are preserved across combined querysets:
q1 = User.query.filter(is_active=True).only("id", "email")
q2 = User.query.filter(is_staff=True).defer("last_login")
rows = await q1.union(q2).order_by("email").values(["id", "email"])
Saffier applies ordering, offset, and limit to the outer combined result, not to the individual
branch orderings. Add an explicit order_by() when you need deterministic pagination or comparison
semantics.
Row locking¶
Use select_for_update() to request row-level locking when running inside a transaction:
async with database.transaction():
users = await User.query.select_for_update(nowait=True).all()
Select related¶
Returns a QuerySet that will “follow” foreign-key relationships, selecting additional related-object data when it executes its query.
This is a performance booster which results in a single more complex query but means
later use of foreign-key relationships won’t require database queries.
A simple query:
profiles = await Profile.query.select_related("user")
Or adding more operations on the top
profiles = await Profile.query.select_related("user").filter(email__icontains="foo").limit(2)
Returning results¶
All¶
Returns all the instances.
users = await User.query.all()
Tip
The all as mentioned before it automatically executed by Saffier if not provided and it can also be aggregated with other queryset operations.
Save¶
This is a classic operation that is very useful depending on which operations you need to perform. Used to save an existing object in the database. Slighly different from the update and simpler to read.
await User.query.create(is_active=True, email="foo@bar.com")
user = await User.query.get(email="foo@bar.com")
user.email = "bar@foo.com"
await user.save()
Now a more unique, yet possible scenario with a save. Imagine you need to create an exact copy of an object and store it in the database. These cases are more common than you think but this is for example purposes only.
await User.query.create(is_active=True, email="foo@bar.com", name="John Doe")
user = await User.query.get(email="foo@bar.com")
# User(id=1)
# Making a quick copy
user.id = None
new_user = await user.save()
# user(id=2)
Create¶
Used to create model instances.
await User.query.create(is_active=True, email="foo@bar.com")
await User.query.create(is_active=False, email="bar@foo.com")
await User.query.create(is_active=True, email="foo@bar.com", first_name="Foo", last_name="Bar")
Bulk get or create¶
Creates missing rows and reuses existing rows when matching unique_fields.
users = await User.query.bulk_get_or_create(
[
{"name": "Alice", "language": "English"},
{"name": "Bob", "language": "Portuguese"},
],
unique_fields=["name", "language"],
)
Alias available: bulk_select_or_insert.
Delete¶
Used to delete rows and return the number of deleted records.
deleted = await User.query.filter(email="foo@bar.com").delete()
To execute per-instance delete hooks/signals during queryset deletion, use:
deleted = await User.query.filter(is_active=False).delete(use_models=True)
Or directly in the instance.
user = await User.query.get(email="foo@bar.com")
deleted = await user.delete()
Use raw_delete() when you want database-level deletion without model-level delete hooks:
deleted = await User.query.filter(is_active=False).raw_delete()
Update¶
You can update model instances by calling this operator.
await User.query.filter(email="foo@bar.com").update(email="bar@foo.com")
Or directly in the instance.
user = await User.query.get(email="foo@bar.com")
await user.update(email="bar@foo.com")
Or not very common but also possible, update all rows in a table.
user = await User.query.update(email="bar@foo.com")
Get¶
Obtains a single record from the database.
user = await User.query.get(email="foo@bar.com")
You can mix the queryset returns with this operator as well.
user = await User.query.filter(email="foo@bar.com").get()
First¶
When you need to return the very first result from a queryset.
user = await User.query.first()
You can also apply filters when needed.
Last¶
When you need to return the very last result from a queryset.
user = await User.query.last()
You can also apply filters when needed.
Exists¶
Returns a boolean confirming if a specific record exists.
exists = await User.query.filter(email="foo@bar.com").exists()
exists = await User.query.exists(email__isnull=True)
Count¶
Returns an integer with the total of records.
total = await User.query.count()
total = await User.query.count(email__icontains="@example.com")
Contains¶
Returns true if the QuerySet contains the provided object.
user = await User.query.create(email="foo@bar.com")
exists = await User.query.contains(instance=user)
Values¶
Returns the model results in a dictionary like format.
await User.query.create(name="John" email="foo@bar.com")
# All values
user = User.query.values()
users == [
{"id": 1, "name": "John", "email": "foo@bar.com"},
]
# Only the name
user = User.query.values("name")
users == [
{"name": "John"},
]
# Or as a list
# Only the name
user = User.query.values(["name"])
users == [
{"name": "John"},
]
# Exclude some values
user = User.query.values(exclude=["id"])
users == [
{"name": "John", "email": "foo@bar.com"},
]
The values() can also be combined with filter, only, exclude as per usual.
Parameters:
- fields - Fields of values to return.
- exclude - Fields to exclude from the return.
- exclude_none - Boolean flag indicating if the fields with
Noneshould be excluded.
Values list¶
Returns the model results in a tuple like format.
await User.query.create(name="John" email="foo@bar.com")
# All values
user = User.query.values_list()
users == [
(1, "John" "foo@bar.com"),
]
# Only the name
user = User.query.values_list("name")
users == [
("John",),
]
# Or as a list
# Only the name
user = User.query.values_list(["name"])
users == [
("John",),
]
# Exclude some values
user = User.query.values(exclude=["id"])
users == [
("John", "foo@bar.com"),
]
# Flattened
user = User.query.values_list("email", flat=True)
users == [
"foo@bar.com",
]
The values_list() can also be combined with filter, only, exclude as per usual.
Parameters:
- fields - Fields of values to return.
- exclude - Fields to exclude from the return.
- exclude_none - Boolean flag indicating if the fields with
Noneshould be excluded. - flat - Boolean flag indicating the results should be flattened.
Only¶
Returns the results containing only the fields in the query and nothing else.
await User.query.create(name="John" email="foo@bar.com")
user = await User.query.only("name")
Warning
You can only use only() or defer() but not both combined or a QuerySetError is raised.
Defer¶
Returns the results containing all the fields but the ones you want to exclude.
await User.query.create(name="John" email="foo@bar.com")
user = await User.query.defer("name")
Warning
You can only use only() or defer() but not both combined or a QuerySetError is raised.
Get or none¶
When querying a model and do not want to raise a ObjectNotFound and
instead returns a None.
user = await User.query.get_or_none(id=1)
Useful methods¶
Get or create¶
When you need get an existing model instance from the matching query. If exists, returns or creates a new one in case of not existing.
Returns a tuple of instance and boolean created.
user, created = await User.query.get_or_create(email="foo@bar.com", defaults={
"is_active": False, "first_name": "Foo"
})
This will query the User model with the email as the lookup key. If it doesn't exist, then it
will use that value with the defaults provided to create a new instance.
Warning
Since the get_or_create() is doing a get internally, it can also raise a
MultipleObjectsReturned.
Update or create¶
When you need to update an existing model instance from the matching query. If exists, returns or creates a new one in case of not existing.
Returns a tuple of instance and boolean created.
user, created = await User.query.update_or_create(email="foo@bar.com", defaults={
"is_active": False, "first_name": "Foo"
})
This will query the User model with the email as the lookup key. If it doesn't exist, then it
will use that value with the defaults provided to create a new instance.
Warning
Since the get_or_create() is doing a get internally, it can also raise a
MultipleObjectsReturned.
Bulk create¶
When you need to create many instances in one go, or in bulk.
await User.query.bulk_create([
{"email": "foo@bar.com", "first_name": "Foo", "last_name": "Bar", "is_active": True},
{"email": "bar@foo.com", "first_name": "Bar", "last_name": "Foo", "is_active": True},
])
Bulk update¶
When you need to update many instances in one go, or in bulk.
await User.query.bulk_create([
{"email": "foo@bar.com", "first_name": "Foo", "last_name": "Bar", "is_active": True},
{"email": "bar@foo.com", "first_name": "Bar", "last_name": "Foo", "is_active": True},
])
users = await User.query.all()
for user in users:
user.is_active = False
await User.query.bulk_update(users, fields=['is_active'])
Operators¶
There are sometimes the need of adding some extra conditions like AND, or OR or even the NOT
into your queries and therefore Saffier provides a simple integration with those.
Saffier provides the and_, or_ and not_ operators directly for you to use, although this ones come with a slighly different approach.
For all the examples, let us use the model below.
import saffier
from saffier import Database, Registry
database = Database("sqlite:///db.sqlite")
models = Registry(database=database)
class User(saffier.Model):
first_name: str = saffier.CharField(max_length=50, null=True)
email: str = saffier.EmailField(max_lengh=100, null=True)
class Meta:
registry = models
SQLAlchemy style¶
Since Saffier is built on the top of SQL Alchemy core, that also means we can also use directly that same functionality within our queries.
In other words, uses the SQLAlchemy style.
Warning
The or_, and_ and not_ do not work with related operations and only
directly with the model itself.
This might sound confusing so let us see some examples.
AND¶
As the name suggests, you want to add the AND explicitly.
import saffier
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the and_
await User.query.filter(
saffier.and_(User.columns.name == "Adam", User.columns.email == "adam@saffier.dev"),
)
As mentioned before, applying the SQLAlchemy style also means you can do this.
import saffier
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the and_
await User.query.filter(
saffier.and_(
User.columns.email.contains("saffier"),
)
)
And you can do nested querysets like multiple filters.
import saffier
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the and_
await User.query.filter(saffier.and_(User.columns.name == "Adam")).filter(
saffier.and_(User.columns.email == "adam@saffier.dev")
)
OR¶
The same principle as the and_ but applied to the OR.
import saffier
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the or_
await User.query.filter(
saffier.or_(User.columns.name == "Adam", User.columns.email == "adam@saffier.dev"),
)
As mentioned before, applying the SQLAlchemy style also means you can do this.
import saffier
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the or_
await User.query.filter(
saffier.or_(
User.columns.email.contains("saffier"),
)
)
And you can do nested querysets like multiple filters.
import saffier
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the or_
await User.query.filter(saffier.or_(User.columns.name == "Adam")).filter(
saffier.or_(User.columns.email == "adam@saffier.dev")
)
NOT¶
This is simple and direct, this is where you apply the NOT.
import saffier
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the not_
await User.query.filter(saffier.not_(User.columns.name == "Adam"))
As mentioned before, applying the SQLAlchemy style also means you can do this.
import saffier
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the not_
await User.query.filter(
saffier.not_(
User.columns.email.contains("saffier"),
)
)
And you can do nested querysets like multiple filters.
import saffier
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
await User.query.create(name="John", email="john@example.com")
# Query using the not_
await User.query.filter(saffier.not_(User.columns.name == "Adam")).filter(
saffier.not_(User.columns.email.contains("saffier"))
)
Saffier Style¶
This is the most common used scenario where you can use the related for your queries and all the great functionalities of Saffier while using the operands.
Tip
The same way you apply the filters for the queries using the related, this can also be done with the Saffier style but the same cannot be said for the SQLAlchemy style. So if you want to leverage the full power of Saffier, it is advised to go Saffier style.
AND¶
The AND operand with the syntax is the same as using the filter or any queryset
operatator but for visualisation purposes this is also available in the format of and_.
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the and_
await User.query.and_(email__icontains="saffier")
With multiple parameters.
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the and_
await User.query.and_(name="Adam", email="adam@saffier.dev")
And you can do nested querysets like multiple filters.
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the and_
await User.query.filter(name="Adam").and_(email="adam@saffier.dev")
OR¶
The same principle as the and_ but applied to the OR.
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the or_
await User.query.or_(name="Adam", email="adam@saffier.dev")
With multiple or_ or nultiple parametes in the same or_
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the multiple or_
await User.query.or_(email__icontains="saffier").or_(name__icontains="a")
# Query using the or_ with multiple fields
await User.query.or_(email__icontains="saffier", name__icontains="a")
And you can do nested querysets like multiple filters.
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the or_
await User.query.or_(name="Adam").filter(email="adam@saffier.dev")
NOT¶
The not_ as the same principle as the exclude and like the and, for
representation purposes, Saffier also has that function.
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the not_
await User.query.not_(name="Adam")
With multiple not_.
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
# Query using the not_
await User.query.not_(email__icontains="saffier").not_(name__icontains="a")
And you can do nested querysets like multiple filters.
# Create some records
await User.query.create(name="Adam", email="adam@saffier.dev")
await User.query.create(name="Eve", email="eve@saffier.dev")
await User.query.create(name="John", email="john@example.com")
# Query using the not_
await User.query.filter(email__icontains="saffier").not_(name__iexact="Adam")
Internally, the not_ is calling the exclude and applying the operators so this is
more for cosmetic purposes than anything else, really.
Blocking Queries¶
What happens if you want to use Saffier with a blocking operation? So by blocking means sync.
For instance, Flask does not support natively async and Saffier is an async agnotic ORM and you
probably would like to take advantage of Saffier but you want without doing a lot of magic behind.
Well, Saffier also supports the run_sync functionality that allows you to run the queries in
blocking mode with ease!
How to use¶
You simply need to use the run_sync functionality from Saffier and make it happen almost immediatly.
from saffier import run_sync
All the available functionalities of Saffier run within this wrapper without extra syntax.
Let us see some examples.
Async mode
await User.query.all()
await User.query.filter(name__icontains="example")
await User.query.create(name="Saffier")
With run_sync
from saffier import run_sync
run_sync(User.query.all())
run_sync(User.query.filter(name__icontains="example"))
run_sync(User.query.create(name="Saffier"))
If synchronous code also needs to manage registry connection lifecycle, wrap it with
Registry.with_async_env():
with models.with_async_env():
run_sync(models.create_all())
run_sync(User.query.create(name="Saffier"))