aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSerhiy Storchaka <storchaka@gmail.com>2018-12-22 11:18:40 +0200
committerGitHub <noreply@github.com>2018-12-22 11:18:40 +0200
commit8ac658114dec4964479baecfbc439fceb40eaa79 (patch)
treee66c4c3beda293a6fdf01763306697d15d0af157 /.gitattributes
parent bpo-22703: IDLE: Improve Code Context and Zoom Height menu labels (GH-11214) (diff)
downloadcpython-8ac658114dec4964479baecfbc439fceb40eaa79.tar.gz
cpython-8ac658114dec4964479baecfbc439fceb40eaa79.tar.bz2
cpython-8ac658114dec4964479baecfbc439fceb40eaa79.zip
bpo-30455: Generate all token related code and docs from Grammar/Tokens. (GH-10370)
"Include/token.h", "Lib/token.py" (containing now some data moved from "Lib/tokenize.py") and new files "Parser/token.c" (containing the code moved from "Parser/tokenizer.c") and "Doc/library/token-list.inc" (included in "Doc/library/token.rst") are now generated from "Grammar/Tokens" by "Tools/scripts/generate_token.py". The script overwrites files only if needed and can be used on the read-only sources tree. "Lib/symbol.py" is now generated by "Tools/scripts/generate_symbol_py.py" instead of been executable itself. Added new make targets "regen-token" and "regen-symbol" which are now dependencies of "regen-all". The documentation contains now strings for operators and punctuation tokens.
Diffstat (limited to '.gitattributes')
-rw-r--r--.gitattributes4
1 files changed, 4 insertions, 0 deletions
diff --git a/.gitattributes b/.gitattributes
index 16237bb2b3a..c9a54fbd472 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -55,3 +55,7 @@ Include/opcode.h linguist-generated=true
Python/opcode_targets.h linguist-generated=true
Objects/typeslots.inc linguist-generated=true
Modules/unicodedata_db.h linguist-generated=true
+Doc/library/token-list.inc linguist-generated=true
+Include/token.h linguist-generated=true
+Lib/token.py linguist-generated=true
+Parser/token.c linguist-generated=true