Enable Abstract Syntax Tree Source Files via Roslyn
Now that Roslyn is available (or will be available), it is time for Visual Studio to start thinking in files that represent Abstract Syntax Trees (ASTs), or .ast files. .ast files are the actual source files that are checked into source control. When an .ast file is loaded into Visual Studio, the developer's "formatting preference configuration" is applied to the .ast file to display the file to the preference of the developer.
Formatting preference configurations would include things like:
- Spaces or Tabs for tabbing
- Braces on same line as method declaration or below
- Spaces around symbols
ReSharper has a great deal of formatting configuration, but this affects the actual source code that gets checked in. This idea formats the "view" of the .ast file (the "actual" source code that gets checked into source control) after it is loaded in Visual Studio.
This paradigm fixes the problem of the re-occurring (and frustrating) issue of formatting code to a particular team's or organization's (or really, a bossy, opinionated manager's/lead/architect's) code formatting requirements (which everyone always disagrees with in some way), when in fact every developer has their own preferred method or way of formatting their code.
By thinking in .ast files, teams get working code checked into source control (with passing tests running against libraries produced from the .ast files), and developers (finally) view their code the way they want and prefer to do it. Everyone wins (and is happy, and wants to work with everyone else forever and ever, amen).
Of course, teams and organizations can provide "guidance" formatting configurations, but developers can take those and modify them to their wishes.
It's really a good idea. And I'm not just saying this because I thought of it. Visual Studio should do this. :) Way overdue.
Unless you enjoy bickering with teammates about the "best" way to format code, please vote for this!
Rudi Larno (Avantida) commented
One advantage that I have not seen mentioned yet is that an Annotated AST could carry a lot more information about the transformations that have been done to the code base. Giving Source Control systems a lot more knowledge about the history of the code base. Today it has to rely on text-based comparison. (We create intelligent systems for just about anything, but code is just still 'text', really?)
One could incorporate 'higher level' functions such as
'rename variable from foo to bar in scope function IsFooBar'
'extract variable foo to member Foo'
which could help source control be more intelligent about how to merge code. (e.g. https://www.semanticmerge.com/)
Developers Win! commented
If anyone is interested, this vote and others like it that aim for key improvements to Visual Studio are being tracked on Developers Win! The first report can be found here:
Future weekly reports will be found here:
As of today, the Visual Studio Improvements category (where this vote is tracked) has 223 combined points. This report will be useful in seeing how many votes occurred over what period of time. Please continue sharing this idea and make sure that our voices are heard!
Hello Administrators, please close this vote and merge it into this one: http://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/10020390-enable-roslyn-powered-code-views
It does a much better job explaining the issue, and has a blog post with pretty pictures to outline the process. Thank you. :)
Again, the issues mentioned would be solved by having the stored format would be just as customizable as the viewed format. The stored format even could be a 100% StyleCop compliant (which dictates how that Start and Stop should be organized/located in your code, btw -- something you may or may not agree with) code file. When you open that stored file, you actually open your own *view* of it, so it would view to YOU exactly how YOU like it... The Start and Stop would be exactly where YOU would like it to be in YOUR version of the file. When you save it, it gets saved and stored as the file format as configured and designed by the team/repo owner (which, again, could be a 100% StyleCop compliant file for the sake of simplicity in demonstrating and communicating this idea). Everyone wins. Which is the point of this feature/vote, as currently only one person "wins", while everyone else ... doesn't.
Roslyn really opens a lot of doors here. This would be a great implementation and use of it. :)
Roy T. commented
Aside from the already mentioned problems with versioning and everyone needing Visual Studio to view/edit source files I also do not like this idea because it removes a lot of meaning from the source code.
I think we all group members, properties, and methods in a class in a way that conveys meaning. For example most of us would put the Start() and Stop() method below each other as someone who is interested in looking at the Start method might also be interested in the Stop method.
Source code generated from a formatting-less format cannot contain this information, or any other information that conveys meaning through ordering and indenting and would thus make something that is hard to understand (code) even harder to understand.
Yes this is definitely an ingrained problem, and just in teams that use Visual Studio, but pretty much everywhere. It's a bit innovative, I admit, and it will take time for change and integration.
However... if Cobol's/Fortran's editors offered something like this back in the day, we would all be using it as well and wouldn't be giving it a second thought. :P
To me this point is moot. I have never been in a dev team where the choice of factoring was up to the end user, and I doubt this will change tomorrow even if this feature would work perfectly well. There is a reason why everyone in a team should be using the same factoring and formatting options...
Now thinking about this some more, I guess .ast files would be a bit of a misnomer. Going back to the idea of "strongly-structured" source files... Those could be files that are formatted to be 100% StyleCop-compliant (along with any rules that an organization would like to remove/add).
The process we really need here is:
1) .ast files are really "strongly-structured" 100% StyleCop-compliant and formatted .cs/.vb files. These are what is checked into source control.
2) When an .ast file is loaded in Visual Studio, an AST is loaded and created from it.
3) The AST from step 2 is then loaded and formatted based off the local developer's preferences. This then creates a local .cs/.vb "view" file that is stored locally (or better yet in the developer's OneDrive).
When a developer saves the .cs/.vb file, Visual Studio attempts to build an AST from it. If it does this successfully, it then saves this AST back to the .ast file as a "strongly-structured" 100% StyleCop-compliant and formatted .cs/.vb file. This trick here would to ensure that it is merge/source-control friendly (shouldn't be a problem) during this process.
I think I like this better. :)
That way, we're always working in C#/VB and we can view the code any time from anywhere using any editor. The only real difference is the internal formatting/translation process that occurs when the file is loaded in Visual Studio.
It is very irritating and unfortunate that you cannot edit a suggestion/idea after you create it! Otherwise I would update it with this new material. Bummer.
Those are good points. Obviously if I had all the answers, I would be making this instead of suggesting it. :)
I have two thoughts around this:
1) .ast files are just serialized AST trees in Roslyn. So they could be serialized as Xaml files which would make them designer-friendly. The designer would then be responsible for processing the Xaml and rendering it in a way that makes sense to the developer. In fact, with this new paradigm, we now have the ability to render it as code (as traditional C#/VB -- exactly how you see it now in Visual Studio) or something completely different altogether. Conceivably, teams (and companies) could build their own renderers for the ASTs now in visually cool and interesting ways.
2) .ast files could still be saved as .cs/.vb files in source control. The difference here now is that the "source" file is formatted in a particular way that is not known to the developer, and is "strongly-structured" based off this formatting. That is, it's still C#/VB. When the file is loaded into Visual Studio, an intermediate process occurs where an AST is created and then what is displayed to the developer is that strongly-structured .cs/.vb file formatted based on the developer's formatting preferences.
Again, I don't have all the answers, and I am just suggesting what comes to mind. The idea here is that what gets checked into source control isn't necessarily unreadable (or binary).
Also keep in mind that this isn't meant to be required technology that everyone is expected to port over to immediately. It's just another way of sourcing files, and really a more exact way of describing the "code." Teams/companies/organizations can continue to treat their code the traditional (and contentious) way if they wish.
If #1 is used above, then you do have a point with viewing code on GitHub. Instead of seeing C#, users would see an Xml/Xaml/Data file and not code. There would have to be some additional thought around this to make this work nicely.
However, much more difficult problems have been solved in the history of computer science. :)
Thanks for the feedback! This is a fun problem to think about.
This is an interesting proposal, here are my two nits.
1. People who do not use Visual Studio cannot view files anymore. Do we want all the code on github to be unreadable?
2. There will be no way to view diffs in source control. Comparing two ast files could have first party support, but again we leave everyone else out with this option.
I am suspicious of changing the paradigm in C# such that your source file are not the "source of truth". I now need to rely entirely on the IDE to do everything for me perfectly. What is there is a bug and the ast file gets messed up? Will I manually edit it? Will I have arguments among my peers about the style guides for edited ast files?
It is similar, but not. The difference is that you have the "source" file (the .ast file) and then the "view" of that file (the "new" .cs/.vb file, or maybe .ast.cs/.ast.vb). The view file is the "formatted" file and would be on the developer's machine and what they interact with. Whenever the view file is saved, a new AST is created and saved to the .ast file. The .ast file is what is they used to compile an assembly.
The formatting can be configured to occur per keystroke and/or manually. A caching system could be created around each .ast file so that these differences you speak of can be account for, assuming you mean differences on a per-file basis (as opposed to a per-developer basis, which is exactly what this solution solves over-all).
Jan Kučera commented
That sounds like visually similar to the document being auto-formatted on every open. This means potentially losing some formatting and also does not allow for differences in single file.