glibc/scripts
Siddhesh Poyarekar f2144b7874 Script to generate ChangeLog-like output from git log
Co-authored-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Joseph Myers <joseph@codesourcery.com>

The utility of a ChangeLog file has been discussed in various mailing
list threads and GNU Tools Cauldrons in the past years and the general
consensus is that while the file may have been very useful in the past
when revision control did not exist or was not as powerful as it is
today, it's current utility is fast diminishing.  Further, the
ChangeLog format gets in the way of modernisation of processes since
it almost always results in rewriting of a commit, thus preventing use
of any code review tools to automatically manage patches in the glibc
project.

There is consensus in the glibc community that documentation of why a
change was done (i.e. a detailed description in a git commit) is more
useful than what changed (i.e. a ChangeLog entry) since the latter can
be deduced from the patch.  The GNU community would however like to
keep the option of ascertaining what changed through a ChangeLog-like
output and as a compromise, it was proposed that a script be developed
that generates this output.

The script below is the result of these discussions.  This script
takes two git revisions references as input and generates the git log
between those revisions in a form that resembles a ChangeLog.  Its
capabilities and limitations are listed in a comment in the script.
On a high level it is capable of parsing C code and telling what
changed at the top level, but not within constructs such as functions.

Design
------

At a high level, the script analyses the raw output of a VCS, parses
the source files that have changed and attempts to determine what
changed.  The script driver needs three distinct components to be
fully functional for a repository:

- A vcstocl_quirks.py file that helps it parse weird patterns in
  sources that may result from preprocessor defines.
- A VCS plugin backend; the git backend is implemented for glibc
- A programming language parser plugin.  C is currently implemented.

Additional programming language parsers can be added to give more
detailed output for changes in those types of files.

For input in languages other than those that have a parser, the script
only identifies if a file has been added, removed, modified,
permissions changed, etc. but cannot understand the change in content.

The C Parser
------------

The C parser is capable of parsing C programs with preprocessor macros
in place, as if they were part of the language.  This presents some
challenges with parsing code that expands macros on the fly and to
help work around that, a vcstocl_quirks.py file has transformations to
ease things.

The C parser currently can identify macro definitions and scopes and
all global and static declarations and definitions.  It cannot parse
(and compare) changes inside functions yet, it could be a future
enhancement if the need for it arises.

Testing
-------

The script has been tested with the glibc repository up to glibc-2.29
and also in the past with emacs.  While it would be ideal to have
something like this in a repository like gnulib, that should not be a
bottleneck for glibc to start using this, so this patch proposes to
add these scripts into glibc.

And here is (hopefully!) one of the last ChangeLog entries we'd have
to write for glibc:

	* scripts/gitlog_to_changelog.py: New script to auto-generate
	ChangeLog.
	* scripts/vcs_to_changelog/frontend_c.py: New file.
	* scripts/vcs_to_changelog/misc_util.py: New file.
	* scripts/vcs_to_changelog/vcs_git.py: New file.
	* scripts/vcs_to_changelog/vcstocl_quirks.py: Likewise.
2019-09-20 11:46:52 -07:00
..
vcs_to_changelog Script to generate ChangeLog-like output from git log 2019-09-20 11:46:52 -07:00
abi-versions.awk Remove bitrotten --enable-oldest-abi (bug 6652). 2014-09-16 17:45:03 +00:00
abilist.awk scripts/abilist.awk: Handle special _end symbol for Hurd 2018-11-30 22:22:57 +01:00
backport-support.sh Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
build-many-glibcs.py Use Linux 5.3 in build-many-glibcs.py. 2019-09-18 15:13:33 +00:00
check-c++-types.sh Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
check-execstack.awk check-execstack: Permit sysdeps to xfail some libs 2018-07-20 03:28:14 +02:00
check-initfini.awk Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
check-installed-headers.sh Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
check-local-headers.sh Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
check-localplt.awk Extend local PLT reference check 2015-07-29 11:58:06 -07:00
check-obsolete-constructs.py Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
check-textrel.awk Do check-textrel test using readelf rather than a build-time C program. 2012-05-01 13:27:11 -07:00
check-wrapper-headers.py Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
config-uname.sh Rejigger header generation for default uname implementation. 2010-08-24 11:56:52 -07:00
config.guess Update miscellaneous files from upstream sources. 2019-01-01 00:52:59 +00:00
config.sub Update miscellaneous files from upstream sources. 2019-01-01 00:52:59 +00:00
cpp Make shebang interpreter directives consistent 2016-01-07 04:03:21 -05:00
cross-test-ssh.sh Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
documented.sh Make shebang interpreter directives consistent 2016-01-07 04:03:21 -05:00
evaluate-test.sh Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
firstversions.awk Fix bug in firstversions.awk version range handling. 2012-01-28 12:02:44 -05:00
gen-as-const.py Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
gen-libc-abis Make shebang interpreter directives consistent 2016-01-07 04:03:21 -05:00
gen-libc-modules.awk Auto-generate libc-modules.h 2014-11-19 12:16:00 +05:30
gen-posix-conf-vars.awk Remove uses of sprintf in gen-posix-conf-vars.awk 2015-01-02 11:16:35 +05:30
gen-rrtypes.py Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
gen-sorted.awk Update copyright dates with scripts/update-copyrights. 2019-01-01 00:11:28 +00:00
gen-tunables.awk Fix range check in do_tunable_update_val 2017-09-27 02:44:09 +05:30
gitlog_to_changelog.py Script to generate ChangeLog-like output from git log 2019-09-20 11:46:52 -07:00
glibcextract.py Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
haveversions.awk Don't build libnsl for new ABIs 2018-09-24 10:23:10 +02:00
install-sh Update miscellaneous files from upstream sources. 2018-12-06 17:21:47 +00:00
lib-names.awk Clean up gnu/lib-names.h generation (bug 14171). 2014-09-26 17:33:04 +00:00
list-fixed-bugs.py Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
list-sources.sh Update scripts/list-sources.sh for ports repository merge. 2014-06-26 21:30:07 +00:00
localplt.awk Support PLT and GOT references in local PIC check 2015-10-14 06:00:02 -07:00
merge-test-results.sh Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
mkinstalldirs Update miscellaneous files from upstream sources. 2018-12-06 17:21:47 +00:00
move-if-change Update miscellaneous files from upstream sources. 2019-01-01 00:52:59 +00:00
output-format.sed 2004-08-13 Daniel Jacobowitz <dan@debian.org> 2004-08-14 06:54:39 +00:00
pylint Implement benchmarking script in python 2014-03-21 17:32:50 +05:30
pylintrc pylintrc: disable reports 2015-11-11 13:41:57 -05:00
rellns-sh Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
soversions.awk Remove bitrotten --enable-oldest-abi (bug 6652). 2014-09-16 17:45:03 +00:00
sysd-rules.awk sysd-rules: Cut down the number of rtld-% pattern rules 2016-09-20 10:41:05 +02:00
test_printers_common.py Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
test_printers_exceptions.py Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
test-installation.pl Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
update-abilist.sh Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
update-copyrights Prefer https to http for gnu.org and fsf.org URLs 2019-09-07 02:43:31 -07:00
versionlist.awk Update copyright dates with scripts/update-copyrights. 2019-01-01 00:11:28 +00:00
versions.awk Update copyright dates with scripts/update-copyrights. 2019-01-01 00:11:28 +00:00