From f2804372a36e1a86e0001f5da53c3c42ee53d04c Mon Sep 17 00:00:00 2001 From: Yehoshua Sandler Date: Mon, 31 Mar 2025 15:37:29 -0500 Subject: [PATCH] doc: update readme on parser --- README.md | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 76c70ba..bb825ab 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,28 @@ Core program to interpret query language strings into structured data, and back again. +## Data Structure Philosophy + +We are operating off of the philosophy that the first class data is SQL Statement stings. + +From these strings we derive all structured data types to represent those SQL statements. +Whether it be CRUD or schema operations. + +Our all of these structs will have to implement the `Query` interface + +```go +type Query interface { + GetFullSql() string +} +``` + +So ever struct we create from SQL will need to be able to provide a full and valid SQL +statement of itself. + +These structs are then where we are able to alter their fields programatically to create +new statements altogether. + + ## SQL Tokens We are currently using DataDog's SQL Tokenizer `sqllexer` to scan through SQL strings. @@ -43,6 +65,16 @@ const ( ``` -Based on these different token types we will be able to parse out the details of said query into -structs to represent them. From those structs we will be able to modify their details any way that -we choose and reconstruct them into valid SQL statements. +These are an OK generalizer to start with when trying to parse out SQL, but can not be used +without conditional logic that checks what the actual keywords are. + +Currently we scan through the strings to tokenize it. When steping through the tokens we try +to determine the type of query we are working with. At that point we assume the over structure +of the rest of the of the statement to fit a particular format, then parse out the details of +the statement into the struct correlating to its data type. + +Checkout the function `ParseSelectStatement` from `q/select.go` as an example. + +## Improvement Possibilities + +- want to to cut down as many `scan`s as possible by injecting functional dependencies