I want to implement a formatted SQL function on the front end. Among them, SQL may contain some mybatis-specific syntax, such as ‘#{var}’.
The input is SQL compressed into one line; the output is formatted SQL.
The figure below is an example of input and output:
figure of input and output example
input example:
select distinct a.name_order as "nameorder", #{authId} as "authid", trim(authType) as "authType", concat('org-', a.org_icon) as "iconCls" from table_user a left join (select b.obj_type, b.obj_id from table_object a left join table_auth_data b on a.auth_id = b.id where a.auth = aa) b on (b.obj_type = trim(a.auth_type) and b.object_id = aa) where a.parent is null and trim(c.org_id) = #{org_Id} order by a.order asc
However, when I add the nested concat() function to the where part of the SQL statement, the parsing will fail, such as:
errer 1:
input
select distinct a.name_order as "nameorder", #{authId} as "authid", trim(authType) as "authType", concat('org-', a.org_icon) as "iconCls" from table_user a left join (select b.obj_type, b.obj_id from table_object a left join table_auth_data b on a.auth_id = b.id where a.auth = concat('a', concat('b', 'c'))) b on (b.obj_type = trim(a.auth_type) and b.object_id = aa) where a.parent is null and trim(c.org_id) = #{org_Id} order by a.order asc
The error message is very long:
nearley.js:343 Uncaught Error: Parse error at token: ) at line 1 column 309
Unexpected CLOSE_PAREN token: {"type":"CLOSE_PAREN","raw":")","text":")","start":308}. Instead, I was expecting to see one of the following:
A PROPERTY_ACCESS_OPERATOR token based on:
property_access → atomic_expression _ ● %PROPERTY_ACCESS_OPERATOR _ property_access$subexpression$1
atomic_expression$subexpression$1 → ● property_access
atomic_expression → ● atomic_expression$subexpression$1
asteriskless_andless_expression$subexpression$1 → ● atomic_expression
asteriskless_andless_expression → ● asteriskless_andless_expression$subexpression$1
asteriskless_free_form_sql$subexpression$1 → ● asteriskless_andless_expression
asteriskless_free_form_sql → ● asteriskless_free_form_sql$subexpression$1
free_form_sql$subexpression$1 → ● asteriskless_free_form_sql
free_form_sql → ● free_form_sql$subexpression$1
other_clause$ebnf$1 → other_clause$ebnf$1 ● free_form_sql
other_clause → %RESERVED_CLAUSE ● other_clause$ebnf$1
clause$subexpression$1 → ● other_clause
clause → ● clause$subexpression$1
expressions_or_clauses$ebnf$2 → expressions_or_clauses$ebnf$2 ● clause
expressions_or_clauses → expressions_or_clauses$ebnf$1 ● expressions_or_clauses$ebnf$2
statement → ● expressions_or_clauses statement$subexpression$1
main$ebnf$1 → main$ebnf$1 ● statement
main → ● main$ebnf$1
A LINE_COMMENT token based on:
comment → ● %LINE_COMMENT
_$ebnf$1 → _$ebnf$1 ● comment
_ → ● _$ebnf$1
property_access → atomic_expression ● _ %PROPERTY_ACCESS_OPERATOR _ property_access$subexpression$1
......Due to character limit, some error messages are omitted here.
A RESERVED_JOIN token based on:
keyword$subexpression$1 → ● %RESERVED_JOIN
keyword → ● keyword$subexpression$1
atomic_expression$subexpression$1 → ● keyword
atomic_expression → ● atomic_expression$subexpression$1
asteriskless_andless_expression$subexpression$1 → ● atomic_expression
asteriskless_andless_expression → ● asteriskless_andless_expression$subexpression$1
asteriskless_free_form_sql$subexpression$1 → ● asteriskless_andless_expression
asteriskless_free_form_sql → ● asteriskless_free_form_sql$subexpression$1
free_form_sql$subexpression$1 → ● asteriskless_free_form_sql
free_form_sql → ● free_form_sql$subexpression$1
other_clause$ebnf$1 → other_clause$ebnf$1 ● free_form_sql
other_clause → %RESERVED_CLAUSE ● other_clause$ebnf$1
clause$subexpression$1 → ● other_clause
clause → ● clause$subexpression$1
expressions_or_clauses$ebnf$2 → expressions_or_clauses$ebnf$2 ● clause
expressions_or_clauses → expressions_or_clauses$ebnf$1 ● expressions_or_clauses$ebnf$2
statement → ● expressions_or_clauses statement$subexpression$1
main$ebnf$1 → main$ebnf$1 ● statement
main → ● main$ebnf$1
at T.O.feed (nearley.js:343:27)
at Object.parse (createParser.ts:33:34)
at CA.parse (Formatter.ts:37:49)
at CA.format (Formatter.ts:31:22)
at tA (sqlFormatter.ts:96:57)
at Module.sA (sqlFormatter.ts:69:10)
at formatSQL (mybatis-xml - 副本.html:68:33)
at HTMLButtonElement.<anonymous> (mybatis-xml - 副本.html:62:30)
============================================
error 2:
input
select distinct a.name_order as "nameorder", #{authId} as "authid", trim(authType) as "authType", concat('org-', a.org_icon) as "iconCls" from table_user a left join (select b.obj_type, b.obj_id from table_object a left join table_auth_data b on a.auth_id = b.id where a.auth = concat(trim(a), concat('b', 'c'))) b on (b.obj_type = trim(a.auth_type) and b.object_id = aa) where a.parent is null and trim(c.org_id) = #{org_Id} order by a.order asc
The error message:
nearley.js:343 Uncaught Error: Parse error at token: ) at line 1 column 313
Unexpected CLOSE_PAREN token: {"type":"CLOSE_PAREN","raw":")","text":")","start":312}. Instead, I was expecting to see one of the following:
A PROPERTY_ACCESS_OPERATOR token based on:
property_access → atomic_expression _ ● %PROPERTY_ACCESS_OPERATOR _ property_access$subexpression$1
atomic_expression$subexpression$1 → ● property_access
atomic_expression → ● atomic_expression$subexpression$1
asteriskless_andless_expression$subexpression$1 → ● atomic_expression
asteriskless_andless_expression → ● asteriskless_andless_expression$subexpression$1
asteriskless_free_form_sql$subexpression$1 → ● asteriskless_andless_expression
asteriskless_free_form_sql → ● asteriskless_free_form_sql$subexpression$1
free_form_sql$subexpression$1 → ● asteriskless_free_form_sql
free_form_sql → ● free_form_sql$subexpression$1
other_clause$ebnf$1 → other_clause$ebnf$1 ● free_form_sql
other_clause → %RESERVED_CLAUSE ● other_clause$ebnf$1
clause$subexpression$1 → ● other_clause
clause → ● clause$subexpression$1
expressions_or_clauses$ebnf$2 → expressions_or_clauses$ebnf$2 ● clause
expressions_or_clauses → expressions_or_clauses$ebnf$1 ● expressions_or_clauses$ebnf$2
statement → ● expressions_or_clauses statement$subexpression$1
main$ebnf$1 → main$ebnf$1 ● statement
main → ● main$ebnf$1
A LINE_COMMENT token based on:
comment → ● %LINE_COMMENT
_$ebnf$1 → _$ebnf$1 ● comment
_ → ● _$ebnf$1
property_access → atomic_expression ● _ %PROPERTY_ACCESS_OPERATOR _ property_access$subexpression$1
atomic_expression$subexpression$1 → ● property_access
atomic_expression → ● atomic_expression$subexpression$1
asteriskless_andless_expression$subexpression$1 → ● atomic_expression
asteriskless_andless_expression → ● asteriskless_andless_expression$subexpression$1
asteriskless_free_form_sql$subexpression$1 → ● asteriskless_andless_expression
asteriskless_free_form_sql → ● asteriskless_free_form_sql$subexpression$1
free_form_sql$subexpression$1 → ● asteriskless_free_form_sql
free_form_sql → ● free_form_sql$subexpression$1
other_clause$ebnf$1 → other_clause$ebnf$1 ● free_form_sql
other_clause → %RESERVED_CLAUSE ● other_clause$ebnf$1
clause$subexpression$1 → ● other_clause
clause → ● clause$subexpression$1
expressions_or_clauses$ebnf$2 → expressions_or_clauses$ebnf$2 ● clause
expressions_or_clauses → expressions_or_clauses$ebnf$1 ● expressions_or_clauses$ebnf$2
statement → ● expressions_or_clauses statement$subexpression$1
main$ebnf$1 → main$ebnf$1 ● statement
main → ● main$ebnf$1
......Due to character limit, some error messages are omitted here.
at T.O.feed (nearley.js:343:27)
at Object.parse (createParser.ts:33:34)
at CA.parse (Formatter.ts:37:49)
at CA.format (Formatter.ts:31:22)
at tA (sqlFormatter.ts:96:57)
at Module.sA (sqlFormatter.ts:69:10)
at formatSQL (mybatis-xml - 副本.html:68:33)
at HTMLButtonElement.<anonymous> (mybatis-xml - 副本.html:62:30)
T.O.feed @ nearley.js:343
parse @ createParser.ts:33
parse @ Formatter.ts:37
format @ Formatter.ts:31
tA @ sqlFormatter.ts:96
sA @ sqlFormatter.ts:69
formatSQL @ mybatis-xml - 副本.html:68
===========================================================
Complete code:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>SQL parse</title>
<style>
.container {
display: flex;
flex-direction: column;
align-items: center;
justify-content: space-around;
margin-top: 10px;
}
.xml-container {
padding: 10px;
width: 75%;
overflow: auto;
}
.xml-container pre {
font-size: 18px;
min-height: 40px;
border: 1px solid #ccc;
padding: 10px;
overflow-y: auto;
}
</style>
</head>
<body>
<div class="container">
<div class="xml-container">
<pre id="originalXml" contenteditable></pre>
</div>
<button id="parseButton">parse SQL</button>
<div class="xml-container">
<h3>parsed data</h3>
<pre id="parsedData" style="font-size: 18px;"></pre>
</div>
</div>
<script src="https://cdn.jsdelivr.net/npm/sql-formatter/dist/sql-formatter.min.js"></script>
<script>
var parseButton = document.getElementById("parseButton");
var originalXmlPre = document.getElementById("originalXml");
var loopParsedDataPre = document.getElementById("parsedData");
parseButton.addEventListener("click", function () {
var xmlString = originalXmlPre.textContent;
var parsedData = formatSQL(xmlString);
loopParsedDataPre.textContent = parsedData;
});
function formatSQL(sql) {
return sqlFormatter.format(sql, {
language: 'sql',
paramTypes: {
custom: [
{ regex: String.raw`#{(.+?)}`},
{ regex: String.raw`trim(.+?)`, flags: 'i'},
{ regex: String.raw`concat(.+?)`, flags: 'i'},
{ regex: String.raw`concat(.+?)`, flags: 'i'},
{ regex: String.raw`concat.+?concat.+?))`, flags: 'i'},
{ regex: String.raw`concat(.+?)concat(.+?)`, flags: 'i'},
],
},
});
}
</script>
</body>
</html>
my attempt:
I tried to use the custom regular function of sqlFormatter to solve the problem, but at best it could only solve a simple concat(). It completely failed for nested concat() and concat() containing trim().
Regular definition code:
function formatSQL(sql) {
return sqlFormatter.format(sql, {
language: 'sql',
paramTypes: {
custom: [
{ regex: String.raw`#{(.+?)}`},
{ regex: String.raw`trim(.+?)`, flags: 'i'},
{ regex: String.raw`concat(.+?)`, flags: 'i'},
{ regex: String.raw`concat(.+?)`, flags: 'i'},
{ regex: String.raw`concat.+?concat.+?))`, flags: 'i'},
{ regex: String.raw`concat(.+?)concat(.+?)`, flags: 'i'},
],
},
});
}