Script to generate ChangeLog-like output from git log
Co-authored-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Joseph Myers <joseph@codesourcery.com>
The utility of a ChangeLog file has been discussed in various mailing
list threads and GNU Tools Cauldrons in the past years and the general
consensus is that while the file may have been very useful in the past
when revision control did not exist or was not as powerful as it is
today, it's current utility is fast diminishing. Further, the
ChangeLog format gets in the way of modernisation of processes since
it almost always results in rewriting of a commit, thus preventing use
of any code review tools to automatically manage patches in the glibc
project.
There is consensus in the glibc community that documentation of why a
change was done (i.e. a detailed description in a git commit) is more
useful than what changed (i.e. a ChangeLog entry) since the latter can
be deduced from the patch. The GNU community would however like to
keep the option of ascertaining what changed through a ChangeLog-like
output and as a compromise, it was proposed that a script be developed
that generates this output.
The script below is the result of these discussions. This script
takes two git revisions references as input and generates the git log
between those revisions in a form that resembles a ChangeLog. Its
capabilities and limitations are listed in a comment in the script.
On a high level it is capable of parsing C code and telling what
changed at the top level, but not within constructs such as functions.
Design
------
At a high level, the script analyses the raw output of a VCS, parses
the source files that have changed and attempts to determine what
changed. The script driver needs three distinct components to be
fully functional for a repository:
- A vcstocl_quirks.py file that helps it parse weird patterns in
sources that may result from preprocessor defines.
- A VCS plugin backend; the git backend is implemented for glibc
- A programming language parser plugin. C is currently implemented.
Additional programming language parsers can be added to give more
detailed output for changes in those types of files.
For input in languages other than those that have a parser, the script
only identifies if a file has been added, removed, modified,
permissions changed, etc. but cannot understand the change in content.
The C Parser
------------
The C parser is capable of parsing C programs with preprocessor macros
in place, as if they were part of the language. This presents some
challenges with parsing code that expands macros on the fly and to
help work around that, a vcstocl_quirks.py file has transformations to
ease things.
The C parser currently can identify macro definitions and scopes and
all global and static declarations and definitions. It cannot parse
(and compare) changes inside functions yet, it could be a future
enhancement if the need for it arises.
Testing
-------
The script has been tested with the glibc repository up to glibc-2.29
and also in the past with emacs. While it would be ideal to have
something like this in a repository like gnulib, that should not be a
bottleneck for glibc to start using this, so this patch proposes to
add these scripts into glibc.
And here is (hopefully!) one of the last ChangeLog entries we'd have
to write for glibc:
* scripts/gitlog_to_changelog.py: New script to auto-generate
ChangeLog.
* scripts/vcs_to_changelog/frontend_c.py: New file.
* scripts/vcs_to_changelog/misc_util.py: New file.
* scripts/vcs_to_changelog/vcs_git.py: New file.
* scripts/vcs_to_changelog/vcstocl_quirks.py: Likewise.
2019-09-20 18:36:55 +00:00
|
|
|
# VCSToChangeLog Quirks for the GNU C Library.
|
|
|
|
|
|
2020-01-01 00:14:33 +00:00
|
|
|
# Copyright (C) 2019-2020 Free Software Foundation, Inc.
|
Script to generate ChangeLog-like output from git log
Co-authored-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Joseph Myers <joseph@codesourcery.com>
The utility of a ChangeLog file has been discussed in various mailing
list threads and GNU Tools Cauldrons in the past years and the general
consensus is that while the file may have been very useful in the past
when revision control did not exist or was not as powerful as it is
today, it's current utility is fast diminishing. Further, the
ChangeLog format gets in the way of modernisation of processes since
it almost always results in rewriting of a commit, thus preventing use
of any code review tools to automatically manage patches in the glibc
project.
There is consensus in the glibc community that documentation of why a
change was done (i.e. a detailed description in a git commit) is more
useful than what changed (i.e. a ChangeLog entry) since the latter can
be deduced from the patch. The GNU community would however like to
keep the option of ascertaining what changed through a ChangeLog-like
output and as a compromise, it was proposed that a script be developed
that generates this output.
The script below is the result of these discussions. This script
takes two git revisions references as input and generates the git log
between those revisions in a form that resembles a ChangeLog. Its
capabilities and limitations are listed in a comment in the script.
On a high level it is capable of parsing C code and telling what
changed at the top level, but not within constructs such as functions.
Design
------
At a high level, the script analyses the raw output of a VCS, parses
the source files that have changed and attempts to determine what
changed. The script driver needs three distinct components to be
fully functional for a repository:
- A vcstocl_quirks.py file that helps it parse weird patterns in
sources that may result from preprocessor defines.
- A VCS plugin backend; the git backend is implemented for glibc
- A programming language parser plugin. C is currently implemented.
Additional programming language parsers can be added to give more
detailed output for changes in those types of files.
For input in languages other than those that have a parser, the script
only identifies if a file has been added, removed, modified,
permissions changed, etc. but cannot understand the change in content.
The C Parser
------------
The C parser is capable of parsing C programs with preprocessor macros
in place, as if they were part of the language. This presents some
challenges with parsing code that expands macros on the fly and to
help work around that, a vcstocl_quirks.py file has transformations to
ease things.
The C parser currently can identify macro definitions and scopes and
all global and static declarations and definitions. It cannot parse
(and compare) changes inside functions yet, it could be a future
enhancement if the need for it arises.
Testing
-------
The script has been tested with the glibc repository up to glibc-2.29
and also in the past with emacs. While it would be ideal to have
something like this in a repository like gnulib, that should not be a
bottleneck for glibc to start using this, so this patch proposes to
add these scripts into glibc.
And here is (hopefully!) one of the last ChangeLog entries we'd have
to write for glibc:
* scripts/gitlog_to_changelog.py: New script to auto-generate
ChangeLog.
* scripts/vcs_to_changelog/frontend_c.py: New file.
* scripts/vcs_to_changelog/misc_util.py: New file.
* scripts/vcs_to_changelog/vcs_git.py: New file.
* scripts/vcs_to_changelog/vcstocl_quirks.py: Likewise.
2019-09-20 18:36:55 +00:00
|
|
|
#
|
|
|
|
|
# The GNU C Library is free software; you can redistribute it and/or
|
|
|
|
|
# modify it under the terms of the GNU Lesser General Public
|
|
|
|
|
# License as published by the Free Software Foundation; either
|
|
|
|
|
# version 2.1 of the License, or (at your option) any later version.
|
|
|
|
|
#
|
|
|
|
|
# The GNU C Library is distributed in the hope that it will be useful,
|
|
|
|
|
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
|
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
|
# Lesser General Public License for more details.
|
|
|
|
|
#
|
|
|
|
|
# You should have received a copy of the GNU Lesser General Public
|
|
|
|
|
# License along with the GNU C Library; if not, see
|
|
|
|
|
# <http://www.gnu.org/licenses/>.
|
|
|
|
|
|
|
|
|
|
from frontend_c import Frontend
|
|
|
|
|
from vcs_to_changelog import ProjectQuirks
|
|
|
|
|
import re
|
|
|
|
|
|
|
|
|
|
class GlibcProjectQuirks(ProjectQuirks):
|
|
|
|
|
repo = 'git'
|
|
|
|
|
|
|
|
|
|
IGNORE_LIST = [
|
|
|
|
|
'ChangeLog',
|
|
|
|
|
'sysdeps/x86_64/dl-trampoline.h'
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
MACRO_QUIRKS = \
|
|
|
|
|
[{'orig': r'ElfW\((\w+)\)', 'sub': r'\1__ELF_NATIVE_CLASS_t'},
|
|
|
|
|
{'orig': r'(libc_freeres_fn)\s*\((\w+)\)', 'sub': r'static void \1__\2 (void)'},
|
|
|
|
|
{'orig': r'(IMPL)\s*\((\w+), .*\)$', 'sub': r'static void \1__\2 (void) {}'},
|
|
|
|
|
{'orig': r'__(BEGIN|END)_DECLS', 'sub': r''},
|
|
|
|
|
{'orig': 'weak_function', 'sub': '__attribute__ ((weak))'},
|
|
|
|
|
{'orig': r'ATTRIBUTE_(CONST|MALLOC|PURE|FORMAT)',
|
|
|
|
|
'sub': r'__attribute__ ((\1))'},
|
|
|
|
|
{'orig': r'__THROW', 'sub': r'__attribute__ ((__nothrow__ __LEAF))'},
|
|
|
|
|
{'orig': r'__THROWNL', 'sub': r'__attribute__ ((__nothrow__))'},
|
2020-01-16 14:49:07 +00:00
|
|
|
{'orig': r'__nonnull \(\(([^)]+)\)\)',
|
|
|
|
|
'sub': r'__attribute__ ((__nonnull__ \1))'},
|
Script to generate ChangeLog-like output from git log
Co-authored-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Joseph Myers <joseph@codesourcery.com>
The utility of a ChangeLog file has been discussed in various mailing
list threads and GNU Tools Cauldrons in the past years and the general
consensus is that while the file may have been very useful in the past
when revision control did not exist or was not as powerful as it is
today, it's current utility is fast diminishing. Further, the
ChangeLog format gets in the way of modernisation of processes since
it almost always results in rewriting of a commit, thus preventing use
of any code review tools to automatically manage patches in the glibc
project.
There is consensus in the glibc community that documentation of why a
change was done (i.e. a detailed description in a git commit) is more
useful than what changed (i.e. a ChangeLog entry) since the latter can
be deduced from the patch. The GNU community would however like to
keep the option of ascertaining what changed through a ChangeLog-like
output and as a compromise, it was proposed that a script be developed
that generates this output.
The script below is the result of these discussions. This script
takes two git revisions references as input and generates the git log
between those revisions in a form that resembles a ChangeLog. Its
capabilities and limitations are listed in a comment in the script.
On a high level it is capable of parsing C code and telling what
changed at the top level, but not within constructs such as functions.
Design
------
At a high level, the script analyses the raw output of a VCS, parses
the source files that have changed and attempts to determine what
changed. The script driver needs three distinct components to be
fully functional for a repository:
- A vcstocl_quirks.py file that helps it parse weird patterns in
sources that may result from preprocessor defines.
- A VCS plugin backend; the git backend is implemented for glibc
- A programming language parser plugin. C is currently implemented.
Additional programming language parsers can be added to give more
detailed output for changes in those types of files.
For input in languages other than those that have a parser, the script
only identifies if a file has been added, removed, modified,
permissions changed, etc. but cannot understand the change in content.
The C Parser
------------
The C parser is capable of parsing C programs with preprocessor macros
in place, as if they were part of the language. This presents some
challenges with parsing code that expands macros on the fly and to
help work around that, a vcstocl_quirks.py file has transformations to
ease things.
The C parser currently can identify macro definitions and scopes and
all global and static declarations and definitions. It cannot parse
(and compare) changes inside functions yet, it could be a future
enhancement if the need for it arises.
Testing
-------
The script has been tested with the glibc repository up to glibc-2.29
and also in the past with emacs. While it would be ideal to have
something like this in a repository like gnulib, that should not be a
bottleneck for glibc to start using this, so this patch proposes to
add these scripts into glibc.
And here is (hopefully!) one of the last ChangeLog entries we'd have
to write for glibc:
* scripts/gitlog_to_changelog.py: New script to auto-generate
ChangeLog.
* scripts/vcs_to_changelog/frontend_c.py: New file.
* scripts/vcs_to_changelog/misc_util.py: New file.
* scripts/vcs_to_changelog/vcs_git.py: New file.
* scripts/vcs_to_changelog/vcstocl_quirks.py: Likewise.
2019-09-20 18:36:55 +00:00
|
|
|
{'orig': r'([^_])attribute_(\w+)', 'sub': r'\1__attribute__ ((\2))'},
|
|
|
|
|
{'orig': r'^attribute_(\w+)', 'sub': r'__attribute__ ((\1))'}]
|
|
|
|
|
|
|
|
|
|
def __init__(self, debug):
|
|
|
|
|
self.debug = debug
|
|
|
|
|
''' Build a list of macro calls used for symbol versioning and attributes.
|
|
|
|
|
|
|
|
|
|
glibc uses a set of macro calls that do not end with a semi-colon and hence
|
|
|
|
|
breaks our parser. Identify those calls from include/libc-symbols.h and
|
|
|
|
|
filter them out.
|
|
|
|
|
'''
|
|
|
|
|
with open('include/libc-symbols.h') as macrofile:
|
|
|
|
|
op = macrofile.readlines()
|
|
|
|
|
op = Frontend.remove_comments(self, op)
|
|
|
|
|
self.C_MACROS = [re.sub(r'.*define (\w+).*', r'\1', x[:-1]) for x in op \
|
|
|
|
|
if 'define ' in x]
|
|
|
|
|
|
|
|
|
|
super().__init__()
|
|
|
|
|
|
|
|
|
|
def get_project_quirks(debug):
|
|
|
|
|
''' Accessor function.
|
|
|
|
|
'''
|
|
|
|
|
return GlibcProjectQuirks(debug)
|
