I am just trying out Diesel and am pretty new to Rust. I am noticing when going through the tutorials that the IDE experience (Visual Studio Code in my case) is not providing much assistance (intellisense, navigating to meaningful code) with respect to the code generated by the macros in the models.rs and schema.rs files.
It is my guess that the reason for this is that the Rust Language Server (RLS) is not able to provide the information necessary for intellisense from the code generated by macros. Is this the case?
If so, might a future version of diesel benefit from generating “non touchable” source code rather than the compact “macro expanded” file schema.rs so that a user of the library benefits from both intellisense and the ability to inspect the generated source code to understand what is happening under the hood?
Taking this concept to its ultimate endpoint, might a future version start with a database agnostic schema definition file stored with each migration, use that to create the initial up.sql and down.sql files (that would need to be modifiable), and then generate all source code in a set of files not to be edited by the developer as they would be regenerated when the database agnostic schema definition has changed.
Would this make sense?
Or is there something about “The Rust way” I might be missing?
It is my guess that the reason for this is that the Rust Language Server (RLS) is not able to provide the information necessary for intellisense from the code generated by macros. Is this the case?
Yes rls does not expand macros, therefore it cannot provide completion there. rust-analyzer provides code completion here, so I consider this as bug in rls.
If so, might a future version of diesel benefit from generating “non touchable” source code rather than the compact “macro expanded” file schema.rs so that a user of the library benefits from both intellisense and the ability to inspect the generated source code to understand what is happening under the hood?
Is explained above I consider this as a bug in rls. Additionally I don’t think it’s a good idea to degrade the user experience by workaround bugs in some code completion tools. Doing what you’ve proposed would significantly degrade the user experience for users who write table! definitions by hand. Additionally this would be a big breaking change, nothing that we consider to do at the moment.
Taking this concept to its ultimate endpoint, might a future version start with a database agnostic schema definition file stored with each migration, use that to create the initial up.sql and down.sql files (that would need to be modifiable), and then generate all source code in a set of files not to be edited by the developer as they would be regenerated when the database agnostic schema definition has changed.
Again doing this would remove the ability to use diesel in a for us quite important environment: Use diesel with existing databases.
Additionally diesel is not designed to abstract over different databases, but to explicitly expose the differences between databases, because otherwise our are not really able to fully utilize the underlying database. Again that’s a quite important usecase for us as core team.
Thank you very much for the detailed response and explanation of the primary goals of the diesel libraries.
I must agree that if the primary goals are:
To be able to write table! macros by hand.
Take full advantage of each database’s offerings in specific ways to their offerings.
then all of the choices you have made are prudent.
Hopefully RLS does allow this intellisense moving forward as I believe that would be key for efficiently coding against database models at least compared to other Non-rust offerings.
Please consider my next statements in the context of my extremely limited experience with Rust and diesel.
Does an IDE provide intellisense when using macros such as “table!”? If so, I can certainly see the appeal of this.
As an alternative, could you support 2 different code generation methods? It is my understanding that a macro, such as table!, simply produces rust code that is ultimately fed to the compiler. Could a developer have the option of using the schema.rs file OR a metadata file, such as a schema.yaml file, that preprocessed into a schema_readonly.rs file that had the final source code spelled out? I am so new to these development stacks that I don’t know if there is a YAML editor (vim plugin) that can take a document schema and provide intellisense as you are writing the meta data. If not, then perhaps a different meta driven format would be better.
I appreciate the effort you have put into the diesel library and as far as I can tell, it is the highest regarded database access library in the Rust ecosystem. And I think you should only implement items that align with your ultimate goals. If my suggestions would increase the learning curve and possible reduce the adoption of diesel, then by all means, please steer clear of them.
I am currently trying to figure out what path would have the total minimum cost of development using Rust for application servers that primarily serve database access to web clients. Currently it is between diesel and starting directly from C or C++ MySql connectors and using a technique similar to the ones I have suggested here. These suggestions are based on the next iteration of a system I created in 2007 and the company I work at is still using today for all database access across all programming languages. But the goals of my system are notably different from yours as you have already identified. Code generation, removing hand crafted SQL statements and other possible coding errors, having the compiler catch all errors related to schema changes over time, and automating development are the only parts they share.
Hopefully RLS does allow this intellisense moving forward as I believe that would be key for efficiently coding against database models at least compared to other Non-rust offerings.
Does an IDE provide intellisense when using macros such as “table!”? If so, I can certainly see the appeal of this.
I’m not sure about RLS and their roadmap here, but at least rust-analyzer already does this. So maybe give it a try?
As an alternative, could you support 2 different code generation methods? It is my understanding that a macro, such as table!, simply produces rust code that is ultimately fed to the compiler. Could a developer have the option of using the schema.rs file OR a metadata file, such as a schema.yaml file, that preprocessed into a schema_readonly.rs file that had the final source code spelled out? I am so new to these development stacks that I don’t know if there is a YAML editor (vim plugin) that can take a document schema and provide intellisense as you are writing the meta data. If not, then perhaps a different meta driven format would be better.
In the end you could hand write everything generated by the table! macro, because diesel needs to expose all stuff required to do this. We do not consider everything in there public API (meaning some stuff can change between minor version), but if you are OK with that you could totally write that by hand/generate it by some tool. That said: I do not think diesel should do that for you. If you want to build something like that you can do that on top of diesel. For expanding the table! macro call you could just use cargo expand for example. For writing you migrations in some metalanguage: Again itself diesel will probably not support that at anytime, but it should be simple to build something on top of diesel that does this. See barrel for an example.