String to Column

1

I am implementing an API which shall allow searching for items in different tables by using filters.
A filter is sent as JSON object having following structure:

{
  "<field>": "<operator><value>"
}

field is an existing column in the queried table and the type is thereby known at compile time.

For example fetching users by age above 20 would look like this:

{
  "age": ">20"
}

Currently I am doing this by implementing a trait function for every possible column type which converts the query to a sql statement considering the type. The query is split by a regular expression. This works but is kind of ugly because I cannot check wether the field exists in the database at compile time and I cannot make use of the query builder of diesel.

Therefore I would like to be able to use the Column to map the operators and fields to the diesel dsl. The most rudimentary solution would be to create a hash map or something similiar.

I am lacking of an idea how to implement this, maybe someone can help me?

2

There is no built-in way in diesel to achieve something like that. At some point you need to manually map strings/column name to compile time types. Or to word it differently: You cannot check that an arbitrary string is a valid database column without explicitly writing out that this is the case.
One possible “solution” to sidestep this issue would be to use something like the unreleased diesel-dynamic-schema, that just does not rely on static compile time types.

Currently I am doing this by implementing a trait function for every possible column type which converts the query to a sql statement considering the type. The query is split by a regular expression.

I should add here: The internal representation our query nodes are not part of our public API and can break at any time. You probably not rely on that behavior for anything.

3

I had a look at diesle-dynamic-schema but this would add an unneeded dependency and an unneeded layer of complexity.

For the solution I was thinking about some proc-macro implementation as I already use a derive macro to get my current implementation. By this and some trait implementation I could rely on the normal QueryDsl implementations that are part of the public diesel API. So not much different from what I am doing right now by hand but more elegant.

The point where I am failing is, when I have to call a function of an instance or struct (Column).
Now a {"age": "=20"} currently leads to a where clause having “age = 20” but I would rather call a function or macro which calls “user::age.eq(20)”.

My thought was that this could be done by a hash map which is created at compile time so I have something like user_column_map["age"].eq(20). Or maybe a mapping function so the following is possible: get_user_column("age").eq(20).

But there I am stuck…

4

Such a functionality cannot exist as each column type is it’s own type and rust functions return/a hashmap can only contain values of a single type. Also boxing/dynamic dispatch/BoxableExpression do not help here as columns have likely different sql types.

Really the only way of doing something like this without using a approach like diesel-dynamic-schema is to have a large match statement like this somewhere.:

match column_str {
    "age" => age.apply_filter(your_filter),
    // other columns
    _ => // handle unmatched column error here,
}
5

You mentioned wundergraph as a solution for this quite a while as I asked this question in gitter for the first time.

Maybe you can describe what you had in mind or point me to the relevant code portions?

I think what I am trying to achieve is indeed related what you did with wundergraph but by for not that complex.

6

The basic idea for implementing filter in wundergraph is not that complex. You have a a bunch of traits build the filter structure from the provided graphql request and map that to diesels dsl.

The basic building block are structs like this one

pub struct Eq<T, C>(Option<T>, PhantomData<C>);

where T is the actual type of the provided filter value and C the column type the filter should be applied.

Then there is this trait impl:

impl<C, T, DB> BuildFilter<DB> for Eq<T, C>
where
    C: ExpressionMethods + NonAggregate + Column + QueryFragment<DB> + Default + 'static,
    T: AsExpression<C::SqlType> + ToSql<<C as Expression>::SqlType, DB>,
    T::Expression: NonAggregate + AppearsOnTable<C::Table> + QueryFragment<DB> + 'static,
    DB: Backend + HasSqlType<<C as Expression>::SqlType> + 'static,
    C::Table: 'static,
    operators::Eq<C, <T as AsExpression<C::SqlType>>::Expression>:
        AppearsOnTable<C::Table, SqlType = Bool>,
{
    type Ret = Box<dyn BoxableFilter<C::Table, DB, SqlType = Bool>>;

    fn into_filter(self) -> Option<Self::Ret> {
        let Self(filter, _) = self;
        filter.map(|v| Box::new(C::default().eq(v)) as Box<_>)
    }
}

which describe how to build a diesel expression out of the struct.

Those simple operation structs are then aggregated into a single high level wrapping struct here

pub struct FilterOption<T, C>
where
    T: FilterValue<C>,
{
    eq: Eq<T::RawValue, C>,
    neq: NotEq<T::RawValue, C>,
    eq_any: EqAny<T::RawValue, C>,
    additional: T::AdditionalFilter,
}

where FilterValue is just a helper trait that allows to use a different type for building the operations and allows to provide more operations specific for the type via the additional field.

Those filters are then aggregated here into a final filter expression by chaining them via and. This trait impl is used to construct this data structure from the provided graphql ast.

Those generic data structures are then used to build a specific filters like:

struct FooFilter {
    first_field: FilterOption<RustFieldType, foo::first_column>,
    second_field: FilterOption<RustFieldType, foo::second_column>,
}

(This requires then a manual “deserialize” implementation that maps the provided query string to the corresponding fields (by just matching fields), as pseudo-code something like this:

fn deserialize(value: serde_json::Value) -> FooFilter {
     if let Some(object) = value {
         let mut ret = Self::default();
         for (key, value) in object {
             match key {
                 "first_field" => {
                      ret.first_field =FilterOption::<RustFieldType, foo::first_column>::deserialize(value);
                  },
                 "second_field" => {
                      ret.second_field =FilterOption::<RustFieldType, foo::second_column>::deserialize(value);
                  },
                  _ => todo!("Handle column mismatch")
             }
         }
     } else {
          todo!("Handle error")
     }
}

This implementation works with query strings like: filter: {"first_field": {"eq": 42}, "second_field": {"neq": "Foo"}} where everything is expected to be statically typed.

7

I appreciate your explanation. I think I got the idea and will try to implement something like this.