Yacc/Lex

With a very short introduction, Lex and Yacc are useful tools that you would use if you want to create a compiler, a transpiler or you just want to interpret a formal language.

YaccLexTools is focused on C#. Other tools are available for other programming languages like C/C++, F#, Java, etc.

YaccLexTools translantes Yacc and Lex files in C# code thanks to the tools GPPG and GPLEX and simplify the use of these tools in the C# projects building process.

Time to update

Some years have passed since I have created the YaccLexTools package aimed to simplify the use of Yacc and Lex files in C# projets. Everything is evolved since then and now we have a lot of .NET everywhere! And it's cross platform, too.

So, it's a long time that this upgrade was needed and now is the time. YaccLexTools is having its long awaited upgrade for finally working with .Net Core, .NET and cross platform along with the .NET Framework with the same easy and automated project setup. With this upgrade the package has been promoted to the version 1.0.0 and it is retro compatible with the projects that have used the old version 0.2.x since now.

When you upgrade the package from the Nuget Package Manager, it will try to automatically upgrade the project settings to the new build model so that everything will work fine as before with some fix and improvements.

For some reasons sometimes the update of the settings is not triggered. In these cases it is possible to upgrade the project settings from the Package Manager Console with this cmdlet

PM> Update-YaccLexToolsSettings

Major changes

• The package works with .NET Framework 2.0+, .Net Core 3.1+, .NET 5+ and can be used on every platform supported by .NET.
• The new build system is based on separate packages that provide the MSBuild targets and tooling for compiling .y and .lex files respectively with the tools GPPG and GPLEX.
• Huge simplification of the project settings. No more <Target> definitions and package specific settings and metadata.
• Common classes have been converted from public to internal for avoiding conflicts in case the project references more libraries and each of them have these classes defined.
• Files generated by GPPG and GPLEX will not include anymore sentitive data like the machine name and the username captured at the moment of the generation of files. There is no reason to publish these information in a public repository.

Powershell cmdlets (Windows only)

Cmdlets are still the same, but can be used only in Visual Studio for Windows. For adding a new parser the command is

PM> Add-Parser MyLanguage Sample.NameSpace

The Calculator example can be added as usual with

PM> Add-CalculatorExample

Obsolete cmdlets for removing parser settings have been removed because there are no more settings to remove :)

If you need to remove a parser, just remove the files from the Solution Explorer and it's done.

dotnet-ylt (Cross-platform)

Since cmdlets are not working on macOS and Linux and they do not work with projects other than .NET Framework, it has been necessary to create a new tool that replaces the cmdlets, but has the same features. The tool is cross-platform and can be installed with the command below

dotnet tool install dotnet-ylt --global

Installing dotnet-ylt globally lets you install it just once and not for every project.

Adding a parser to a .NET console application is very easy. Open the terminal in the project folder and type the command

dotnet ylt add-parser -p <parserName>

The parser name is given with the parameter -p or --parser.

For adding the calculator example:

dotnet ylt add-calculator

Both commands support the optional parameter -n or --namespace for creating the parser in the given namespace.

New build system

After adding the calculator example to a .NET 6 console application and after building it, in the Solution Explorer the project looks like this:

The files GplexBuffers.cs and ShiftReduceParserCode.cs are generated during the build process and no more when the parser is added to the project.

The project file becomes as below.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  <ItemGroup Label="Calculator">
    <YaccFile Include="Calculator\Calculator.Language.grammar.y">
      <OutputFile>Calculator\Calculator.Parser.Generated.cs</OutputFile>
      <Arguments>/gplex /nolines</Arguments>
    </YaccFile>
    <None Include="Calculator\Calculator.Parser.Generated.cs">
      <AutoGen>True</AutoGen>
      <DesignTime>True</DesignTime>
      <DependentUpon>Calculator.Language.grammar.y</DependentUpon>
    </None>
    <LexFile Include="Calculator\Calculator.Language.analyzer.lex">
      <OutputFile>Calculator\Calculator.Scanner.Generated.cs</OutputFile>
    </LexFile>
    <None Include="Calculator\Calculator.Scanner.Generated.cs">
      <AutoGen>True</AutoGen>
      <DesignTime>True</DesignTime>
      <DependentUpon>Calculator.Language.analyzer.lex</DependentUpon>
    </None>
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="YaccLexTools" Version="1.0.0" />
  </ItemGroup>
</Project>

It is much more compact in respect to the previous format.

The code below is even more compact and produces the same results. The code above has the advantage to hide the generated files as sub-nodes of their respective source files, but for the moment this works only in Visual Studio for Windows. Maybe a future relase of Visual Studio for Mac will implement this feature. Below, the arguments for the YaccFile are omitted because they have the same default value: /gplex /nolines.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  <ItemGroup Label="Calculator">
    <YaccFile Include="Calculator\Calculator.Language.grammar.y">
      <OutputFile>Calculator\Calculator.Parser.Generated.cs</OutputFile>
    </YaccFile>
    <LexFile Include="Calculator\Calculator.Language.analyzer.lex">
      <OutputFile>Calculator\Calculator.Scanner.Generated.cs</OutputFile>
    </LexFile>
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="YaccLexTools" Version="1.0.0" />
  </ItemGroup>
</Project>

With the hope that these updates are useful to those who are already using the package and to all of those who will use it, see you in the next one!

ernstc

Old posts

• Languages and Translators with Visual Studio
• Yacc/Lex Tools v0.2

References

• Yacc/Lex Tools project repository: github.com/ernstc/YaccLexTools
• GPPG original repository: github.com/k-john-gough/gppg
• GPLEX original repository: github.com/k-john-gough/gplex
• YACC: en.wikipedia.org/wiki/Yacc
• LEX: en.wikipedia.org/wiki/Lex_(software)