to allow full expressions like ``CONSTANT - 1``.)
-Renaming the C functions generated by Argument Clinic
------------------------------------------------------
+Renaming the C functions and variables generated by Argument Clinic
+-------------------------------------------------------------------
Argument Clinic automatically names the functions it generates for you.
Occasionally this may cause a problem, if the generated name collides with
and the impl function would now be named ``pickler_dumper_impl()``.
+Similarly, you may have a problem where you want to give a parameter
+a specific Python name, but that name may be inconvenient in C. Argument
+Clinic allows you to give a parameter different names in Python and in C,
+using the same ``"as"`` syntax::
+
+ /*[clinic input]
+ pickle.Pickler.dump
+
+ obj: object
+ file as file_obj: object
+ protocol: object = NULL
+ *
+ fix_imports: bool = True
+
+Here, the name used in Python (in the signature and the ``keywords``
+array) would be ``file``, but the C variable would be named ``file_obj``.
+
+You can use this to rename the ``self`` parameter too!
+
Converting functions using PyArg_UnpackTuple
--------------------------------------------
(If your function doesn't support keywords, the parsing function
generated will throw an exception if it receives any.)
-The #ifdef trick
-----------------------------------------------
-
-If you're converting a function that isn't available on all platforms,
-there's a trick you can use to make life a little easier. The existing
-code probably looks like this::
-
- #ifdef HAVE_FUNCTIONNAME
- static module_functionname(...)
- {
- ...
- }
- #endif /* HAVE_FUNCTIONNAME */
-
-And then in the ``PyMethodDef`` structure at the bottom you'll have::
-
- #ifdef HAVE_FUNCTIONNAME
- {'functionname', ... },
- #endif /* HAVE_FUNCTIONNAME */
-
-In this scenario, you should change the code to look like the following::
-
- #ifdef HAVE_FUNCTIONNAME
- /*[clinic input]
- module.functionname
- ...
- [clinic start generated code]*/
- static module_functionname(...)
- {
- ...
- }
- #endif /* HAVE_FUNCTIONNAME */
-
-Run Argument Clinic on the code in this state, then refresh the file in
-your editor. Now you'll have the generated code, including the #define
-for the ``PyMethodDef``, like so::
-
- #ifdef HAVE_FUNCTIONNAME
- /*[clinic input]
- ...
- [clinic start generated code]*/
- ...
- #define MODULE_FUNCTIONNAME \
- {'functionname', ... },
- ...
- /*[clinic end generated code: checksum=...]*/
- static module_functionname(...)
- {
- ...
- }
- #endif /* HAVE_FUNCTIONNAME */
-
-Change the #endif at the bottom as follows::
-
- #else
- #define MODULE_FUNCTIONNAME
- #endif /* HAVE_FUNCTIONNAME */
-
-Now you can remove the #ifdefs around the ``PyMethodDef`` structure
-at the end, and replace those three lines with ``MODULE_FUNCTIONNAME``.
-If the function is available, the macro turns into the ``PyMethodDef``
-static value, including the trailing comma; if the function isn't
-available, the macro turns into nothing. Perfect!
-
-(This is the preferred approach for optional functions; in the future,
-Argument Clinic may generate the entire ``PyMethodDef`` structure.)
-
-
Changing and redirecting Clinic's output
----------------------------------------
``output preset`` sets Clinic's output to one of several built-in
preset configurations, as follows:
- ``original``
- Clinic's starting configuration.
+ ``block``
+ Clinic's original starting configuration. Writes everything
+ immediately after the input block.
Suppress the ``parser_prototype``
and ``docstring_prototype``, write everything else to ``block``.
the file was not modified by hand before it gets overwritten.
+The #ifdef trick
+----------------------------------------------
+
+If you're converting a function that isn't available on all platforms,
+there's a trick you can use to make life a little easier. The existing
+code probably looks like this::
+
+ #ifdef HAVE_FUNCTIONNAME
+ static module_functionname(...)
+ {
+ ...
+ }
+ #endif /* HAVE_FUNCTIONNAME */
+
+And then in the ``PyMethodDef`` structure at the bottom the existing code
+will have::
+
+ #ifdef HAVE_FUNCTIONNAME
+ {'functionname', ... },
+ #endif /* HAVE_FUNCTIONNAME */
+
+In this scenario, you should enclose the body of your impl function inside the ``#ifdef``,
+like so::
+
+ #ifdef HAVE_FUNCTIONNAME
+ /*[clinic input]
+ module.functionname
+ ...
+ [clinic start generated code]*/
+ static module_functionname(...)
+ {
+ ...
+ }
+ #endif /* HAVE_FUNCTIONNAME */
+
+Then, remove those three lines from the ``PyMethodDef`` structure,
+replacing them with the macro Argument Clinic generated::
+
+ MODULE_FUNCTIONNAME_METHODDEF
+
+(You can find the real name for this macro inside the generated code.
+Or you can calculate it yourself: it's the name of your function as defined
+on the first line of your block, but with periods changed to underscores,
+uppercased, and ``"_METHODDEF"`` added to the end.)
+
+Perhaps you're wondering: what if ``HAVE_FUNCTIONNAME`` isn't defined?
+The ``MODULE_FUNCTIONNAME_METHODDEF`` macro won't be defined either!
+
+Here's where Argument Clinic gets very clever. It actually detects that the
+Argument Clinic block might be deactivated by the ``#ifdef``. When that
+happens, it generates a little extra code that looks like this::
+
+ #ifndef MODULE_FUNCTIONNAME_METHODDEF
+ #define MODULE_FUNCTIONNAME_METHODDEF
+ #endif /* !defined(MODULE_FUNCTIONNAME_METHODDEF) */
+
+That means the macro always works. If the function is defined, this turns
+into the correct structure, including the trailing comma. If the function is
+undefined, this turns into nothing.
+
+However, this causes one ticklish problem: where should Argument Clinic put this
+extra code when using the "block" output preset? It can't go in the output block,
+because that could be decativated by the ``#ifdef``. (That's the whole point!)
+
+In this situation, Argument Clinic writes the extra code to the "buffer" destination.
+This may mean that you get a complaint from Argument Clinic::
+
+ Warning in file "Modules/posixmodule.c" on line 12357:
+ Destination buffer 'buffer' not empty at end of file, emptying.
+
+When this happens, just open your file, find the ``dump buffer`` block that
+Argument Clinic added to your file (it'll be at the very bottom), then
+move it above the ``PyMethodDef`` structure where that macro is used.
+
+
+
Using Argument Clinic in Python files
-------------------------------------
Tools/Demos
-----------
+- #Issue 20456: Argument Clinic now observes the C preprocessor conditional
+ compilation statements of the C files it parses. When a Clinic block is
+ inside a conditional code, it adjusts its output to match, including
+ automatically generating an empty methoddef macro.
+
+- #Issue 20456: Cloned functions in Argument Clinic now use the correct
+ name, not the name of the function they were cloned from, for text
+ strings inside generated code.
+
+- #Issue 20456: Fixed Argument Clinic's test suite and "--converters" feature.
+
+- #Issue 20456: Argument Clinic now allows specifying different names
+ for a parameter in Python and C, using "as" on the parameter line.
+
- Issue #20326: Argument Clinic now uses a simple, unique signature to
annotate text signatures in docstrings, resulting in fewer false
positives. "self" parameters are also explicitly marked, allowing
[clinic start generated code]*/
PyDoc_STRVAR(curses_window_addch__doc__,
-"addch(self, [x, y,] ch, [attr])\n"
+"addch([x, y,] ch, [attr])\n"
"Paint character ch at (y, x) with attributes attr.\n"
"\n"
" x\n"
static PyObject *
curses_window_addch_impl(PyCursesWindowObject *self, int group_left_1, int x, int y, PyObject *ch, int group_right_1, long attr)
-/*[clinic end generated code: output=e1cdbd4f4e42fc6b input=fe7e3711d5bbf1f6]*/
+/*[clinic end generated code: output=43acb91a5c98f615 input=fe7e3711d5bbf1f6]*/
{
PyCursesWindowObject *cwself = (PyCursesWindowObject *)self;
int coordinates_group = group_left_1;
/*[python input]
class dbmobject_converter(self_converter):
type = "dbmobject *"
- def converter_init(self):
+ def pre_render(self):
+ super().pre_render()
self.name = 'dp'
[python start generated code]*/
-/*[python end generated code: output=da39a3ee5e6b4b0d input=8a69ac1827811128]*/
+/*[python end generated code: output=da39a3ee5e6b4b0d input=6ad536357913879a]*/
static PyObject *
newdbmobject(const char *file, int flags, int mode)
self: dbmobject
key: str(length=True)
- [
- default: object
- ]
+ default: object = None
/
Return the value for key if present, otherwise default.
[clinic start generated code]*/
PyDoc_STRVAR(dbm_dbm_get__doc__,
-"get(self, key, [default])\n"
+"sig=($self, key, default=None)\n"
"Return the value for key if present, otherwise default.");
#define DBM_DBM_GET_METHODDEF \
{"get", (PyCFunction)dbm_dbm_get, METH_VARARGS, dbm_dbm_get__doc__},
static PyObject *
-dbm_dbm_get_impl(dbmobject *dp, const char *key, Py_ssize_clean_t key_length, int group_right_1, PyObject *default_value);
+dbm_dbm_get_impl(dbmobject *dp, const char *key, Py_ssize_clean_t key_length, PyObject *default_value);
static PyObject *
dbm_dbm_get(dbmobject *dp, PyObject *args)
PyObject *return_value = NULL;
const char *key;
Py_ssize_clean_t key_length;
- int group_right_1 = 0;
- PyObject *default_value = NULL;
-
- switch (PyTuple_GET_SIZE(args)) {
- case 1:
- if (!PyArg_ParseTuple(args, "s#:get", &key, &key_length))
- goto exit;
- break;
- case 2:
- if (!PyArg_ParseTuple(args, "s#O:get", &key, &key_length, &default_value))
- goto exit;
- group_right_1 = 1;
- break;
- default:
- PyErr_SetString(PyExc_TypeError, "dbm.dbm.get requires 1 to 2 arguments");
- goto exit;
- }
- return_value = dbm_dbm_get_impl(dp, key, key_length, group_right_1, default_value);
+ PyObject *default_value = Py_None;
+
+ if (!PyArg_ParseTuple(args,
+ "s#|O:get",
+ &key, &key_length, &default_value))
+ goto exit;
+ return_value = dbm_dbm_get_impl(dp, key, key_length, default_value);
exit:
return return_value;
}
static PyObject *
-dbm_dbm_get_impl(dbmobject *dp, const char *key, Py_ssize_clean_t key_length, int group_right_1, PyObject *default_value)
-/*[clinic end generated code: output=31d5180d6b36f1ea input=43a561dc2bd1db3b]*/
+dbm_dbm_get_impl(dbmobject *dp, const char *key, Py_ssize_clean_t key_length, PyObject *default_value)
+/*[clinic end generated code: output=2bbaf9a187f9b6bf input=aecf5efd2f2b1a3b]*/
{
datum dbm_key, val;
- if (!group_right_1)
- default_value = Py_None;
dbm_key.dptr = (char *)key;
dbm_key.dsize = key_length;
check_dbmobject_open(dp);
_opcode.stack_effect -> int
opcode: int
-
- [
- oparg: int
- ]
+ oparg: object = None
/
Compute the stack effect of the opcode.
[clinic start generated code]*/
PyDoc_STRVAR(_opcode_stack_effect__doc__,
-"stack_effect(module, opcode, [oparg])\n"
+"sig=($module, opcode, oparg=None)\n"
"Compute the stack effect of the opcode.");
#define _OPCODE_STACK_EFFECT_METHODDEF \
{"stack_effect", (PyCFunction)_opcode_stack_effect, METH_VARARGS, _opcode_stack_effect__doc__},
static int
-_opcode_stack_effect_impl(PyModuleDef *module, int opcode, int group_right_1, int oparg);
+_opcode_stack_effect_impl(PyModuleDef *module, int opcode, PyObject *oparg);
static PyObject *
_opcode_stack_effect(PyModuleDef *module, PyObject *args)
{
PyObject *return_value = NULL;
int opcode;
- int group_right_1 = 0;
- int oparg = 0;
+ PyObject *oparg = Py_None;
int _return_value;
- switch (PyTuple_GET_SIZE(args)) {
- case 1:
- if (!PyArg_ParseTuple(args, "i:stack_effect", &opcode))
- goto exit;
- break;
- case 2:
- if (!PyArg_ParseTuple(args, "ii:stack_effect", &opcode, &oparg))
- goto exit;
- group_right_1 = 1;
- break;
- default:
- PyErr_SetString(PyExc_TypeError, "_opcode.stack_effect requires 1 to 2 arguments");
- goto exit;
- }
- _return_value = _opcode_stack_effect_impl(module, opcode, group_right_1, oparg);
+ if (!PyArg_ParseTuple(args,
+ "i|O:stack_effect",
+ &opcode, &oparg))
+ goto exit;
+ _return_value = _opcode_stack_effect_impl(module, opcode, oparg);
if ((_return_value == -1) && PyErr_Occurred())
goto exit;
return_value = PyLong_FromLong((long)_return_value);
}
static int
-_opcode_stack_effect_impl(PyModuleDef *module, int opcode, int group_right_1, int oparg)
-/*[clinic end generated code: output=4689140ffda2494a input=056816407c3d4284]*/
+_opcode_stack_effect_impl(PyModuleDef *module, int opcode, PyObject *oparg)
+/*[clinic end generated code: output=4fe636f5db87c0a9 input=2d0a9ee53c0418f5]*/
{
int effect;
+ int oparg_int = 0;
if (HAS_ARG(opcode)) {
- if (!group_right_1) {
+ PyObject *i_object;
+ if (oparg == Py_None) {
PyErr_SetString(PyExc_ValueError,
"stack_effect: opcode requires oparg but oparg was not specified");
return -1;
}
+ i_object = PyNumber_Index(oparg);
+ if (!i_object)
+ return -1;
+ oparg_int = (int)PyLong_AsLong(oparg);
+ if ((oparg_int == -1) && PyErr_Occurred())
+ return -1;
}
- else if (group_right_1) {
+ else if (oparg != Py_None) {
PyErr_SetString(PyExc_ValueError,
"stack_effect: opcode does not permit oparg but oparg was specified");
return -1;
}
- effect = PyCompile_OpcodeStackEffect(opcode, oparg);
+ effect = PyCompile_OpcodeStackEffect(opcode, oparg_int);
if (effect == PY_INVALID_STACK_EFFECT) {
PyErr_SetString(PyExc_ValueError,
"invalid opcode or oparg");
return return_value;
}
+#if defined(HAVE_ZLIB_COPY)
+
PyDoc_STRVAR(zlib_Compress_copy__doc__,
"sig=($self)\n"
"Return a copy of the compression object.");
return zlib_Compress_copy_impl(self);
}
+#endif /* defined(HAVE_ZLIB_COPY) */
+
+#ifndef ZLIB_COMPRESS_COPY_METHODDEF
+ #define ZLIB_COMPRESS_COPY_METHODDEF
+#endif /* !defined(ZLIB_COMPRESS_COPY_METHODDEF) */
+
+#if defined(HAVE_ZLIB_COPY)
+
PyDoc_STRVAR(zlib_Decompress_copy__doc__,
"sig=($self)\n"
"Return a copy of the decompression object.");
return zlib_Decompress_copy_impl(self);
}
+#endif /* defined(HAVE_ZLIB_COPY) */
+
+#ifndef ZLIB_DECOMPRESS_COPY_METHODDEF
+ #define ZLIB_DECOMPRESS_COPY_METHODDEF
+#endif /* !defined(ZLIB_DECOMPRESS_COPY_METHODDEF) */
+
PyDoc_STRVAR(zlib_Decompress_flush__doc__,
"sig=($self, length=DEF_BUF_SIZE)\n"
"Return a bytes object containing any remaining decompressed data.\n"
return return_value;
}
-/*[clinic end generated code: output=ad23316b49faf7e6 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=21556008559f839c input=a9049054013a1b77]*/
converter = 'path_converter'
def converter_init(self, *, allow_fd=False, nullable=False):
- def strify(value):
- return str(int(bool(value)))
-
# right now path_t doesn't support default values.
# to support a default value, you'll need to override initialize().
+ if self.default is not unspecified:
+ fail("Can't specify a default to the path_t converter!")
- assert self.default is unspecified
+ if self.c_default is not None:
+ fail("Can't specify a c_default to the path_t converter!")
self.nullable = nullable
self.allow_fd = allow_fd
+ def pre_render(self):
+ def strify(value):
+ return str(int(bool(value)))
+
+ # add self.py_name here when merging with posixmodule conversion
self.c_default = 'PATH_T_INITIALIZE("{}", {}, {})'.format(
self.function.name,
- strify(nullable),
- strify(allow_fd),
+ strify(self.nullable),
+ strify(self.allow_fd),
)
def cleanup(self):
[python start generated code]*/
-/*[python end generated code: output=da39a3ee5e6b4b0d input=d702d58a8469cc7d]*/
+/*[python end generated code: output=da39a3ee5e6b4b0d input=5c9f456f53244fc3]*/
/*[clinic input]
#endif /* MS_WINDOWS */
+/*[clinic input]
+dump buffer
+[clinic start generated code]*/
+
+#ifndef OS_TTYNAME_METHODDEF
+ #define OS_TTYNAME_METHODDEF
+#endif /* !defined(OS_TTYNAME_METHODDEF) */
+/*[clinic end generated code: output=5d071bbc8f49ea12 input=524ce2e021e4eba6]*/
+
static PyMethodDef posix_methods[] = {
#endif /* HAVE_DYNAMIC_LOADING */
+/*[clinic input]
+dump buffer
+[clinic start generated code]*/
+
+#ifndef _IMP_LOAD_DYNAMIC_METHODDEF
+ #define _IMP_LOAD_DYNAMIC_METHODDEF
+#endif /* !defined(_IMP_LOAD_DYNAMIC_METHODDEF) */
+/*[clinic end generated code: output=d07c1d4a343a9579 input=524ce2e021e4eba6]*/
+
PyDoc_STRVAR(doc_imp,
"(Extremely) low-level import machinery bits as used by importlib and imp.");
_IMP_INIT_FROZEN_METHODDEF
_IMP_IS_BUILTIN_METHODDEF
_IMP_IS_FROZEN_METHODDEF
-#ifdef HAVE_DYNAMIC_LOADING
_IMP_LOAD_DYNAMIC_METHODDEF
-#endif
_IMP__FIX_CO_FILENAME_METHODDEF
{NULL, NULL} /* sentinel */
};
#ifdef __cplusplus
}
#endif
+
import atexit
import collections
import contextlib
+import copy
+import cpp
import functools
import hashlib
import inspect
stop_line = ""
checksum_line = ""
+ def __init__(self, filename):
+ pass
+
@abc.abstractmethod
def render(self, clinic, signatures):
pass
+ def parse_line(self, line):
+ pass
+
def validate(self):
def assert_only_one(attr, *additional_fields):
"""
return tuple(accumulator)
+def strip_leading_and_trailing_blank_lines(s):
+ lines = s.rstrip().split('\n')
+ while lines:
+ line = lines[0]
+ if line.strip():
+ break
+ del lines[0]
+ return '\n'.join(lines)
+
+@functools.lru_cache()
+def normalize_snippet(s, *, indent=0):
+ """
+ Reformats s:
+ * removes leading and trailing blank lines
+ * ensures that it does not end with a newline
+ * dedents so the first nonwhite character on any line is at column "indent"
+ """
+ s = strip_leading_and_trailing_blank_lines(s)
+ s = textwrap.dedent(s)
+ if indent:
+ s = textwrap.indent(s, ' ' * indent)
+ return s
+
+
class CLanguage(Language):
body_prefix = "#"
stop_line = "[{dsl_name} start generated code]*/"
checksum_line = "/*[{dsl_name} end generated code: {arguments}]*/"
+ def __init__(self, filename):
+ super().__init__(filename)
+ self.cpp = cpp.Monitor(filename)
+ self.cpp.fail = fail
+
+ def parse_line(self, line):
+ self.cpp.writeline(line)
+
def render(self, clinic, signatures):
function = None
for o in signatures:
add('"')
return ''.join(text)
- _templates = {}
- # the templates will be run through str.format(),
- # so actual curly-braces need to be doubled up.
- templates_source = """
-__________________________________________________
-
-docstring_prototype
-
-PyDoc_VAR({c_basename}__doc__);
-__________________________________________________
-
-docstring_definition
-
-PyDoc_STRVAR({c_basename}__doc__,
-{docstring});
-__________________________________________________
-
-impl_definition
-
-static {impl_return_type}
-{c_basename}_impl({impl_parameters})
-__________________________________________________
-
-parser_prototype_noargs
-
-static PyObject *
-{c_basename}({self_type}{self_name}, PyObject *Py_UNUSED(ignored))
-__________________________________________________
-
-parser_prototype_meth_o
-
-# SLIGHT HACK
-# METH_O uses {impl_parameters} for the parser!
-
-static PyObject *
-{c_basename}({impl_parameters})
-__________________________________________________
-
-parser_prototype_varargs
-
-static PyObject *
-{c_basename}({self_type}{self_name}, PyObject *args)
-__________________________________________________
-
-parser_prototype_keyword
-
-static PyObject *
-{c_basename}({self_type}{self_name}, PyObject *args, PyObject *kwargs)
-__________________________________________________
-
-parser_prototype_init
-
-static int
-{c_basename}({self_type}{self_name}, PyObject *args, PyObject *kwargs)
-__________________________________________________
-
-parser_definition_simple_no_parsing
-
-{{
- return {c_basename}_impl({impl_arguments});
-}}
-__________________________________________________
-
-parser_definition_start
-
-{{
- {return_value_declaration}
- {declarations}
- {initializers}
-{empty line}
-__________________________________________________
-
-parser_definition_end
-
- {return_conversion}
-
-{exit_label}
- {cleanup}
- return return_value;
-}}
-__________________________________________________
-
-parser_definition_impl_call
-
- {modifications}
- {return_value} = {c_basename}_impl({impl_arguments});
-__________________________________________________
-
-parser_definition_unpack_tuple
-
- if (!PyArg_UnpackTuple(args, "{name}",
- {unpack_min}, {unpack_max},
- {parse_arguments}))
- goto exit;
-__________________________________________________
-
-parser_definition_parse_tuple
-
- if (!PyArg_ParseTuple(args,
- "{format_units}:{name}",
- {parse_arguments}))
- goto exit;
-__________________________________________________
-
-parser_definition_option_groups
- {option_group_parsing}
-
-__________________________________________________
-
-parser_definition_parse_tuple_and_keywords
-
- if (!PyArg_ParseTupleAndKeywords(args, kwargs,
- "{format_units}:{name}", _keywords,
- {parse_arguments}))
- goto exit;
-__________________________________________________
-
-parser_definition_no_positional
-
- if ({self_type_check}!_PyArg_NoPositional("{name}", args))
- goto exit;
-
-__________________________________________________
-
-parser_definition_no_keywords
-
- if ({self_type_check}!_PyArg_NoKeywords("{name}", kwargs))
- goto exit;
-
-__________________________________________________
-
-methoddef_define
-
-#define {methoddef_name} \\
- {{"{name}", (PyCFunction){c_basename}, {methoddef_flags}, {c_basename}__doc__}},
-__________________________________________________
-""".rstrip()
-
- title = ''
- buffer = []
- line = None
- for line in templates_source.split('\n'):
- line = line.rstrip()
- if line.startswith('# '):
- # comment
- continue
- if line.startswith("_____"):
- if not buffer:
- continue
- assert title not in _templates, "defined template twice: " + repr(title)
- buffer = '\n'.join(buffer).rstrip()
- buffer = buffer.replace('{empty line}', '')
- _templates[title] = buffer
- buffer = []
- title = ''
- continue
- if not title:
- if not line:
- continue
- title = line
- continue
- if not (line or buffer):
- # throw away leading blank lines
- continue
- buffer.append(line)
-
- assert not title, 'ensure templates_source ends with ______ (still adding to ' + repr(title) + ")"
-
- del templates_source
- del title
- del buffer
- del line
-
- # for name, value in _templates.items():
- # print(name + ":")
- # pprint.pprint(value)
- # print()
-
def output_templates(self, f):
parameters = list(f.parameters.values())
assert parameters
converters[0].format_unit == 'O' and
not new_or_init)
- # we have to set seven things before we're done:
+ # we have to set these things before we're done:
#
# docstring_prototype
# docstring_definition
# parser_prototype
# parser_definition
# impl_definition
-
- templates = self._templates
+ # cpp_if
+ # cpp_endif
+ # methoddef_ifndef
return_value_declaration = "PyObject *return_value = NULL;"
- methoddef_define = templates['methoddef_define']
+ methoddef_define = normalize_snippet("""
+ #define {methoddef_name} \\
+ {{"{name}", (PyCFunction){c_basename}, {methoddef_flags}, {c_basename}__doc__}},
+ """)
if new_or_init and not f.docstring:
docstring_prototype = docstring_definition = ''
else:
- docstring_prototype = templates['docstring_prototype']
- docstring_definition = templates['docstring_definition']
- impl_definition = templates['impl_definition']
+ docstring_prototype = normalize_snippet("""
+ PyDoc_VAR({c_basename}__doc__);
+ """)
+ docstring_definition = normalize_snippet("""
+ PyDoc_STRVAR({c_basename}__doc__,
+ {docstring});
+ """)
+ impl_definition = normalize_snippet("""
+ static {impl_return_type}
+ {c_basename}_impl({impl_parameters})
+ """)
impl_prototype = parser_prototype = parser_definition = None
+ parser_prototype_keyword = normalize_snippet("""
+ static PyObject *
+ {c_basename}({self_type}{self_name}, PyObject *args, PyObject *kwargs)
+ """)
+
+ parser_prototype_varargs = normalize_snippet("""
+ static PyObject *
+ {c_basename}({self_type}{self_name}, PyObject *args)
+ """)
+
+ # parser_body_fields remembers the fields passed in to the
+ # previous call to parser_body. this is used for an awful hack.
parser_body_fields = ()
def parser_body(prototype, *fields):
nonlocal parser_body_fields
add, output = text_accumulator()
add(prototype)
parser_body_fields = fields
+
fields = list(fields)
- fields.insert(0, 'parser_definition_start')
- fields.append('parser_definition_impl_call')
- fields.append('parser_definition_end')
+ fields.insert(0, normalize_snippet("""
+ {{
+ {return_value_declaration}
+ {declarations}
+ {initializers}
+ """) + "\n")
+ # just imagine--your code is here in the middle
+ fields.append(normalize_snippet("""
+ {modifications}
+ {return_value} = {c_basename}_impl({impl_arguments});
+ {return_conversion}
+
+ {exit_label}
+ {cleanup}
+ return return_value;
+ }}
+ """))
for field in fields:
add('\n')
- add(templates[field])
+ add(field)
return output()
def insert_keywords(s):
flags = "METH_NOARGS"
- parser_prototype = templates['parser_prototype_noargs']
+ parser_prototype = normalize_snippet("""
+ static PyObject *
+ {c_basename}({self_type}{self_name}, PyObject *Py_UNUSED(ignored))
+ """)
parser_definition = parser_prototype
if default_return_converter:
- parser_definition = parser_prototype + '\n' + templates['parser_definition_simple_no_parsing']
+ parser_definition = parser_prototype + '\n' + normalize_snippet("""
+ {{
+ return {c_basename}_impl({impl_arguments});
+ }}
+ """)
else:
parser_definition = parser_body(parser_prototype)
elif meth_o:
flags = "METH_O"
- # impl_definition = templates['parser_prototype_meth_o']
+
+ meth_o_prototype = normalize_snippet("""
+ static PyObject *
+ {c_basename}({impl_parameters})
+ """)
if default_return_converter:
# maps perfectly to METH_O, doesn't need a return converter.
# so we skip making a parse function
# and call directly into the impl function.
impl_prototype = parser_prototype = parser_definition = ''
- impl_definition = templates['parser_prototype_meth_o']
+ impl_definition = meth_o_prototype
else:
- parser_prototype = templates['parser_prototype_meth_o']
+ # SLIGHT HACK
+ # use impl_parameters for the parser here!
+ parser_prototype = meth_o_prototype
parser_definition = parser_body(parser_prototype)
elif has_option_groups:
# in a big switch statement)
flags = "METH_VARARGS"
- parser_prototype = templates['parser_prototype_varargs']
+ parser_prototype = parser_prototype_varargs
- parser_definition = parser_body(parser_prototype, 'parser_definition_option_groups')
+ parser_definition = parser_body(parser_prototype, ' {option_group_parsing}')
elif positional and all_boring_objects:
# positional-only, but no option groups,
# PyArg_UnpackTuple!
flags = "METH_VARARGS"
- parser_prototype = templates['parser_prototype_varargs']
+ parser_prototype = parser_prototype_varargs
- parser_definition = parser_body(parser_prototype, 'parser_definition_unpack_tuple')
+ parser_definition = parser_body(parser_prototype, normalize_snippet("""
+ if (!PyArg_UnpackTuple(args, "{name}",
+ {unpack_min}, {unpack_max},
+ {parse_arguments}))
+ goto exit;
+ """, indent=4))
elif positional:
# positional-only, but no option groups
# we only need one call to PyArg_ParseTuple
flags = "METH_VARARGS"
- parser_prototype = templates['parser_prototype_varargs']
+ parser_prototype = parser_prototype_varargs
- parser_definition = parser_body(parser_prototype, 'parser_definition_parse_tuple')
+ parser_definition = parser_body(parser_prototype, normalize_snippet("""
+ if (!PyArg_ParseTuple(args,
+ "{format_units}:{name}",
+ {parse_arguments}))
+ goto exit;
+ """, indent=4))
else:
# positional-or-keyword arguments
flags = "METH_VARARGS|METH_KEYWORDS"
- parser_prototype = templates['parser_prototype_keyword']
-
- parser_definition = parser_body(parser_prototype, 'parser_definition_parse_tuple_and_keywords')
+ parser_prototype = parser_prototype_keyword
+
+ body = normalize_snippet("""
+ if (!PyArg_ParseTupleAndKeywords(args, kwargs,
+ "{format_units}:{name}", _keywords,
+ {parse_arguments}))
+ goto exit;
+ """, indent=4)
+ parser_definition = parser_body(parser_prototype, normalize_snippet("""
+ if (!PyArg_ParseTupleAndKeywords(args, kwargs,
+ "{format_units}:{name}", _keywords,
+ {parse_arguments}))
+ goto exit;
+ """, indent=4))
parser_definition = insert_keywords(parser_definition)
methoddef_define = ''
if f.kind == METHOD_NEW:
- parser_prototype = templates['parser_prototype_keyword']
+ parser_prototype = parser_prototype_keyword
else:
return_value_declaration = "int return_value = -1;"
- parser_prototype = templates['parser_prototype_init']
+ parser_prototype = normalize_snippet("""
+ static int
+ {c_basename}({self_type}{self_name}, PyObject *args, PyObject *kwargs)
+ """)
fields = list(parser_body_fields)
parses_positional = 'METH_NOARGS' not in flags
assert parses_positional
if not parses_keywords:
- fields.insert(0, 'parser_definition_no_keywords')
+ fields.insert(0, normalize_snippet("""
+ if ({self_type_check}!_PyArg_NoKeywords("{name}", kwargs))
+ goto exit;
+ """, indent=4))
if not parses_positional:
- fields.insert(0, 'parser_definition_no_positional')
+ fields.insert(0, normalize_snippet("""
+ if ({self_type_check}!_PyArg_NoPositional("{name}", args))
+ goto exit;
+ """, indent=4))
parser_definition = parser_body(parser_prototype, *fields)
if parses_keywords:
methoddef_define = methoddef_define.replace('{methoddef_flags}', flags)
+ methoddef_ifndef = ''
+ conditional = self.cpp.condition()
+ if not conditional:
+ cpp_if = cpp_endif = ''
+ else:
+ cpp_if = "#if " + conditional
+ cpp_endif = "#endif /* " + conditional + " */"
+
+ if methoddef_define:
+ methoddef_ifndef = normalize_snippet("""
+ #ifndef {methoddef_name}
+ #define {methoddef_name}
+ #endif /* !defined({methoddef_name}) */
+ """)
+
+
# add ';' to the end of parser_prototype and impl_prototype
# (they mustn't be None, but they could be an empty string.)
assert parser_prototype is not None
"parser_prototype" : parser_prototype,
"parser_definition" : parser_definition,
"impl_definition" : impl_definition,
+ "cpp_if" : cpp_if,
+ "cpp_endif" : cpp_endif,
+ "methoddef_ifndef" : methoddef_ifndef,
}
# make sure we didn't forget to assign something,
add, output = text_accumulator()
data = CRenderData()
- parameters = list(f.parameters.values())
- assert parameters, "We should always have a 'self' at this point!"
-
+ assert f.parameters, "We should always have a 'self' at this point!"
+ parameters = f.render_parameters
converters = [p.converter for p in parameters]
templates = self.output_templates(f)
def _line(self):
self.line_number += 1
- return self.input.pop()
+ line = self.input.pop()
+ self.language.parse_line(line)
+ return line
def parse_verbatim_block(self):
add, output = text_accumulator()
# The callable should not call builtins.print.
return_converters = {}
+clinic = None
class Clinic:
presets_text = """
+preset block
+everything block
+docstring_prototype suppress
+parser_prototype suppress
+cpp_if suppress
+cpp_endif suppress
+methoddef_ifndef buffer
+
preset original
everything block
docstring_prototype suppress
parser_prototype suppress
+cpp_if suppress
+cpp_endif suppress
+methoddef_ifndef buffer
preset file
everything file
d = self.destinations.get
self.field_destinations = collections.OrderedDict((
+ ('cpp_if', d('suppress')),
('docstring_prototype', d('suppress')),
('docstring_definition', d('block')),
('methoddef_define', d('block')),
('impl_prototype', d('block')),
('parser_prototype', d('suppress')),
('parser_definition', d('block')),
+ ('cpp_endif', d('suppress')),
+ ('methoddef_ifndef', d('buffer')),
('impl_definition', d('block')),
))
fail("Can't extract file type for file " + repr(filename))
try:
- language = extensions[extension]()
+ language = extensions[extension](filename)
except KeyError:
fail("Can't identify file type for file " + repr(filename))
self.self_converter = None
self.suppress_signature = suppress_signature
+ self.rendered_parameters = None
+
+ __render_parameters__ = None
+ @property
+ def render_parameters(self):
+ if not self.__render_parameters__:
+ self.__render_parameters__ = l = []
+ for p in self.parameters.values():
+ p = p.copy()
+ p.converter.pre_render()
+ l.append(p)
+ return self.__render_parameters__
+
@property
def methoddef_flags(self):
if self.kind in (METHOD_INIT, METHOD_NEW):
def __repr__(self):
return '<clinic.Function ' + self.name + '>'
+ def copy(self, **overrides):
+ kwargs = {
+ 'name': self.name, 'module': self.module, 'parameters': self.parameters,
+ 'cls': self.cls, 'c_basename': self.c_basename,
+ 'full_name': self.full_name,
+ 'return_converter': self.return_converter, 'return_annotation': self.return_annotation,
+ 'docstring': self.docstring, 'kind': self.kind, 'coexist': self.coexist,
+ 'suppress_signature': self.suppress_signature,
+ }
+ kwargs.update(overrides)
+ f = Function(**kwargs)
+
+ parameters = collections.OrderedDict()
+ for name, value in f.parameters.items():
+ value = value.copy(function=f)
+ parameters[name] = value
+ f.parameters = parameters
+ return f
+
class Parameter:
"""
def is_keyword_only(self):
return self.kind == inspect.Parameter.KEYWORD_ONLY
+ def copy(self, **overrides):
+ kwargs = {
+ 'name': self.name, 'kind': self.kind, 'default':self.default,
+ 'function': self.function, 'converter': self.converter, 'annotation': self.annotation,
+ 'docstring': self.docstring, 'group': self.group,
+ }
+ kwargs.update(overrides)
+ if 'converter' not in overrides:
+ converter = copy.copy(self.converter)
+ converter.function = kwargs['function']
+ kwargs['converter'] = converter
+ return Parameter(**kwargs)
+
+
+
+class LandMine:
+ # try to access any
+ def __init__(self, message):
+ self.__message__ = message
+
+ def __repr__(self):
+ return '<LandMine ' + repr(self.__message__) + ">"
+
+ def __getattribute__(self, name):
+ if name in ('__repr__', '__message__'):
+ return super().__getattribute__(name)
+ # raise RuntimeError(repr(name))
+ fail("Stepped on a land mine, trying to access attribute " + repr(name) + ":\n" + self.__message__)
def add_c_converter(f, name=None):
if ((cls.format_unit != 'O&') and
(cls.format_unit not in legacy_converters)):
legacy_converters[cls.format_unit] = cls
+ if cls.format_unit:
+ legacy_converters[cls.format_unit] = cls
return cls
def add_legacy_c_converter(format_unit, **kwargs):
added_f = f
else:
added_f = functools.partial(f, **kwargs)
- legacy_converters[format_unit] = added_f
+ if format_unit:
+ legacy_converters[format_unit] = added_f
return f
return closure
parameters must be keyword-only.
"""
+ # The C name to use for this variable.
+ name = None
+
+ # The Python name to use for this variable.
+ py_name = None
+
# The C type to use for this variable.
# 'type' should be a Python string specifying the type, e.g. "int".
# If this is a pointer type, the type string should end with ' *'.
signature_name = None
# keep in sync with self_converter.__init__!
- def __init__(self, name, function, default=unspecified, *, c_default=None, py_default=None, annotation=unspecified, **kwargs):
- self.function = function
+ def __init__(self, name, py_name, function, default=unspecified, *, c_default=None, py_default=None, annotation=unspecified, **kwargs):
self.name = name
+ self.py_name = py_name
if default is not unspecified:
if self.default_type and not isinstance(default, (self.default_type, Unknown)):
if annotation != unspecified:
fail("The 'annotation' parameter is not currently permitted.")
+
+ # this is deliberate, to prevent you from caching information
+ # about the function in the init.
+ # (that breaks if we get cloned.)
+ # so after this change we will noisily fail.
+ self.function = LandMine("Don't access members of self.function inside converter_init!")
self.converter_init(**kwargs)
+ self.function = function
def converter_init(self):
pass
data.modifications.append('/* modifications for ' + name + ' */\n' + modifications.rstrip())
# keywords
- data.keywords.append(original_name)
+ data.keywords.append(parameter.name)
# format_units
if self.is_optional() and '|' not in data.format_units:
"""
return ""
+ def pre_render(self):
+ """
+ A second initialization function, like converter_init,
+ called just before rendering.
+ You are permitted to examine self.function here.
+ """
+ pass
+
class bool_converter(CConverter):
type = 'int'
type = None
format_unit = ''
-
def converter_init(self, *, type=None):
+ self.specified_type = type
+
+ def pre_render(self):
f = self.function
default_type, default_name = correct_name_for_self(f)
self.signature_name = default_name
- self.type = type or self.type or default_type
+ self.type = self.specified_type or self.type or default_type
kind = self.function.kind
new_or_init = kind in (METHOD_NEW, METHOD_INIT)
return
if field not in fd:
- fail("Invalid field " + repr(field) + ", must be one of:\n " + ", ".join(valid_fields))
+ fail("Invalid field " + repr(field) + ", must be one of:\n preset push pop print everything " + " ".join(fd))
fd[field] = d
def directive_dump(self, name):
# self.block = self.ClinicOutputBlock(self)
if self.ignore_line(line):
return
+
+ # is it a directive?
+ fields = shlex.split(line)
+ directive_name = fields[0]
+ directive = self.directives.get(directive_name, None)
+ if directive:
+ try:
+ directive(*fields[1:])
+ except TypeError as e:
+ fail(str(e))
+ return
+
self.next(self.state_modulename_name, line)
def state_modulename_name(self, line):
self.indent.infer(line)
- # is it a directive?
- fields = shlex.split(line)
- directive_name = fields[0]
- directive = self.directives.get(directive_name, None)
- if directive:
- try:
- directive(*fields[1:])
- except TypeError as e:
- fail(str(e))
- return
-
# are we cloning?
before, equals, existing = line.rpartition('=')
if equals:
else:
existing_function = None
if not existing_function:
- print("class", cls, "module", module, "exsiting", existing)
+ print("class", cls, "module", module, "existing", existing)
print("cls. functions", cls.functions)
fail("Couldn't find existing function " + repr(existing) + "!")
if not (existing_function.kind == self.kind and existing_function.coexist == self.coexist):
fail("'kind' of function and cloned function don't match! (@classmethod/@staticmethod/@coexist)")
- self.function = Function(name=function_name, full_name=full_name, module=module, cls=cls, c_basename=c_basename,
- return_converter=existing_function.return_converter, kind=existing_function.kind, coexist=existing_function.coexist)
-
- self.function.parameters = existing_function.parameters.copy()
+ self.function = existing_function.copy(name=function_name, full_name=full_name, module=module, cls=cls, c_basename=c_basename, docstring='')
self.block.signatures.append(self.function)
(cls or module).functions.append(self.function)
kwargs = {}
if cls and type == "PyObject *":
kwargs['type'] = cls.typedef
- sc = self.function.self_converter = self_converter(name, self.function, **kwargs)
+ sc = self.function.self_converter = self_converter(name, name, self.function, **kwargs)
p_self = Parameter(sc.name, inspect.Parameter.POSITIONAL_ONLY, function=self.function, converter=sc)
self.function.parameters[sc.name] = p_self
else:
fail("Function " + self.function.name + " has an unsupported group configuration. (Unexpected state " + str(self.parameter_state) + ".a)")
+ # handle "as" for parameters too
+ c_name = None
+ name, have_as_token, trailing = line.partition(' as ')
+ if have_as_token:
+ name = name.strip()
+ if ' ' not in name:
+ fields = trailing.strip().split(' ')
+ if not fields:
+ fail("Invalid 'as' clause!")
+ c_name = fields[0]
+ if c_name.endswith(':'):
+ name += ':'
+ c_name = c_name[:-1]
+ fields[0] = name
+ line = ' '.join(fields)
+
base, equals, default = line.rpartition('=')
if not equals:
base = default
legacy_str = "legacy " if legacy else ""
if name not in dict:
fail('{} is not a valid {}converter'.format(name, legacy_str))
- converter = dict[name](parameter_name, self.function, value, **kwargs)
+ # if you use a c_name for the parameter, we just give that name to the converter
+ # but the parameter object gets the python name
+ converter = dict[name](c_name or parameter_name, parameter_name, self.function, value, **kwargs)
kind = inspect.Parameter.KEYWORD_ONLY if self.keyword_only else inspect.Parameter.POSITIONAL_OR_KEYWORD
return f.docstring
add, output = text_accumulator()
- parameters = list(f.parameters.values())
+ parameters = f.render_parameters
##
## docstring first line
name = p.converter.signature_name or p.name
a = []
- if isinstance(p.converter, self_converter) and not f.suppress_signature:
- # annotate first parameter as being a "self".
- #
- # if inspect.Signature gets this function, and it's already bound,
- # the self parameter will be stripped off.
- #
- # if it's not bound, it should be marked as positional-only.
- a.append('$')
- a.append(name)
+ if isinstance(p.converter, self_converter):
+ if f.suppress_signature:
+ continue
+ else:
+ # annotate first parameter as being a "self".
+ #
+ # if inspect.Signature gets this function, and it's already bound,
+ # the self parameter will be stripped off.
+ #
+ # if it's not bound, it should be marked as positional-only.
+ a.append('$')
+ a.append(name)
+ else:
+ a.append(name)
if p.converter.is_optional():
a.append('=')
value = p.converter.py_default
import unittest
from unittest import TestCase
+
class FakeConverter:
def __init__(self, name, args):
self.name = name
def __init__(self):
self.converters = FakeConvertersDict()
self.legacy_converters = FakeConvertersDict()
- self.language = clinic.CLanguage()
+ self.language = clinic.CLanguage(None)
self.filename = None
self.block_parser = clinic.BlockParser('', self.language)
self.modules = collections.OrderedDict()
+ self.classes = collections.OrderedDict()
clinic.clinic = self
self.name = "FakeClinic"
self.line_prefix = self.line_suffix = ''
# so it woudl spit out an end line for you.
# and since you really already had one,
# the last line of the block got corrupted.
- c = clinic.Clinic(clinic.CLanguage())
+ c = clinic.Clinic(clinic.CLanguage(None))
raw = "/*[clinic]\nfoo\n[clinic]*/"
cooked = c.parse(raw).splitlines()
end_line = cooked[2].rstrip()
class ClinicBlockParserTest(TestCase):
def _test(self, input, output):
- language = clinic.CLanguage()
+ language = clinic.CLanguage(None)
blocks = list(clinic.BlockParser(input, language))
writer = clinic.BlockPrinter(language)
""")
def _test_clinic(self, input, output):
- language = clinic.CLanguage()
+ language = clinic.CLanguage(None)
c = clinic.Clinic(language)
c.parsers['inert'] = InertParser(c)
c.parsers['copy'] = CopyParser(c)
def
[copy start generated code]*/
abc
-/*[copy end generated code: checksum=03cfd743661f07975fa2f1220c5194cbaff48451]*/
+/*[copy end generated code: output=03cfd743661f0797 input=7b18d017f89f61cf]*/
xyz
""", """
verbatim text here
def
[copy start generated code]*/
def
-/*[copy end generated code: checksum=7b18d017f89f61cf17d47f92749ea6930a3f1deb]*/
+/*[copy end generated code: output=7b18d017f89f61cf input=7b18d017f89f61cf]*/
xyz
""")
def test_param(self):
function = self.parse_function("module os\nos.access\n path: int")
self.assertEqual("access", function.name)
- self.assertEqual(1, len(function.parameters))
+ self.assertEqual(2, len(function.parameters))
p = function.parameters['path']
self.assertEqual('path', p.name)
self.assertIsInstance(p.converter, clinic.int_converter)
module os
os.access
follow_symlinks: bool = True
- something_else: str""")
+ something_else: str = ''""")
p = function.parameters['follow_symlinks']
- self.assertEqual(2, len(function.parameters))
+ self.assertEqual(3, len(function.parameters))
self.assertIsInstance(function.parameters['something_else'].converter, clinic.str_converter)
+ def test_param_default_parameters_out_of_order(self):
+ s = self.parse_function_should_fail("""
+module os
+os.access
+ follow_symlinks: bool = True
+ something_else: str""")
+ self.assertEqual(s, """Error on line 0:
+Can't have a parameter without a default ('something_else')
+after a parameter with a default!
+""")
+
def disabled_test_converter_arguments(self):
function = self.parse_function("module os\nos.access\n path: path_t(allow_fd=1)")
p = function.parameters['path']
Perform a stat system call on the given path.""")
self.assertEqual("""
-stat(path)
+sig=($module, path)
Perform a stat system call on the given path.
path
Okay, we're done here.
""")
self.assertEqual("""
-bar(x, y)
+sig=($module, x, y)
This is the documentation for foo.
x
path: str
This/used to break Clinic!
""")
- self.assertEqual("os.stat(path)\n\nThis/used to break Clinic!", function.docstring)
+ self.assertEqual("sig=($module, path)\n\nThis/used to break Clinic!", function.docstring)
def test_c_name(self):
function = self.parse_function("module os\nos.stat as os_stat_fn")
""")
self.assertEqual(s,
('Error on line 0:\n'
- 'Function two_top_groups_on_left has an unsupported group configuration. (Unexpected state 2)\n'))
+ 'Function two_top_groups_on_left has an unsupported group configuration. (Unexpected state 2.b)\n'))
def test_disallowed_grouping__two_top_groups_on_right(self):
self.parse_function_should_fail("""
Docstring
""")
- self.assertEqual("bar()\nDocstring", function.docstring)
- self.assertEqual(0, len(function.parameters))
+ self.assertEqual("sig=($module)\nDocstring", function.docstring)
+ self.assertEqual(1, len(function.parameters)) # self!
def test_illegal_module_line(self):
self.parse_function_should_fail("""
Not at column 0!
""")
self.assertEqual("""
-bar(x, *, y)
+sig=($module, x, *, y)
Not at column 0!
x
path: str
This/used to break Clinic!
""")
- self.assertEqual("stat(path)\nThis/used to break Clinic!", function.docstring)
+ self.assertEqual("sig=($module, path)\nThis/used to break Clinic!", function.docstring)
def test_directive(self):
c = FakeClinic()
--- /dev/null
+import re
+import sys
+
+def negate(condition):
+ """
+ Returns a CPP conditional that is the opposite of the conditional passed in.
+ """
+ if condition.startswith('!'):
+ return condition[1:]
+ return "!" + condition
+
+class Monitor:
+ """
+ A simple C preprocessor that scans C source and computes, line by line,
+ what the current C preprocessor #if state is.
+
+ Doesn't handle everything--for example, if you have /* inside a C string,
+ without a matching */ (also inside a C string), or with a */ inside a C
+ string but on another line and with preprocessor macros in between...
+ the parser will get lost.
+
+ Anyway this implementation seems to work well enough for the CPython sources.
+ """
+
+ is_a_simple_defined = re.compile(r'^defined\s*\(\s*[A-Za-z0-9_]+\s*\)$').match
+
+ def __init__(self, filename=None, *, verbose=False):
+ self.stack = []
+ self.in_comment = False
+ self.continuation = None
+ self.line_number = 0
+ self.filename = filename
+ self.verbose = verbose
+
+ def __repr__(self):
+ return ''.join((
+ '<Monitor ',
+ str(id(self)),
+ " line=", str(self.line_number),
+ " condition=", repr(self.condition()),
+ ">"))
+
+ def status(self):
+ return str(self.line_number).rjust(4) + ": " + self.condition()
+
+ def condition(self):
+ """
+ Returns the current preprocessor state, as a single #if condition.
+ """
+ return " && ".join(condition for token, condition in self.stack)
+
+ def fail(self, *a):
+ if self.filename:
+ filename = " " + self.filename
+ else:
+ filename = ''
+ print("Error at" + filename, "line", self.line_number, ":")
+ print(" ", ' '.join(str(x) for x in a))
+ sys.exit(-1)
+
+ def close(self):
+ if self.stack:
+ self.fail("Ended file while still in a preprocessor conditional block!")
+
+ def write(self, s):
+ for line in s.split("\n"):
+ self.writeline(line)
+
+ def writeline(self, line):
+ self.line_number += 1
+ line = line.strip()
+
+ def pop_stack():
+ if not self.stack:
+ self.fail("#" + token + " without matching #if / #ifdef / #ifndef!")
+ return self.stack.pop()
+
+ if self.continuation:
+ line = self.continuation + line
+ self.continuation = None
+
+ if not line:
+ return
+
+ if line.endswith('\\'):
+ self.continuation = line[:-1].rstrip() + " "
+ return
+
+ # we have to ignore preprocessor commands inside comments
+ #
+ # we also have to handle this:
+ # /* start
+ # ...
+ # */ /* <-- tricky!
+ # ...
+ # */
+ # and this:
+ # /* start
+ # ...
+ # */ /* also tricky! */
+ if self.in_comment:
+ if '*/' in line:
+ # snip out the comment and continue
+ #
+ # GCC allows
+ # /* comment
+ # */ #include <stdio.h>
+ # maybe other compilers too?
+ _, _, line = line.partition('*/')
+ self.in_comment = False
+
+ while True:
+ if '/*' in line:
+ if self.in_comment:
+ self.fail("Nested block comment!")
+
+ before, _, remainder = line.partition('/*')
+ comment, comment_ends, after = remainder.partition('*/')
+ if comment_ends:
+ # snip out the comment
+ line = before.rstrip() + ' ' + after.lstrip()
+ continue
+ # comment continues to eol
+ self.in_comment = True
+ line = before.rstrip()
+ break
+
+ # we actually have some // comments
+ # (but block comments take precedence)
+ before, line_comment, comment = line.partition('//')
+ if line_comment:
+ line = before.rstrip()
+
+ if not line.startswith('#'):
+ return
+
+ line = line[1:].lstrip()
+ assert line
+
+ fields = line.split()
+ token = fields[0].lower()
+ condition = ' '.join(fields[1:]).strip()
+
+ if_tokens = {'if', 'ifdef', 'ifndef'}
+ all_tokens = if_tokens | {'elif', 'else', 'endif'}
+
+ if token not in all_tokens:
+ return
+
+ # cheat a little here, to reuse the implementation of if
+ if token == 'elif':
+ pop_stack()
+ token = 'if'
+
+ if token in if_tokens:
+ if not condition:
+ self.fail("Invalid format for #" + token + " line: no argument!")
+ if token == 'if':
+ if not self.is_a_simple_defined(condition):
+ condition = "(" + condition + ")"
+ else:
+ fields = condition.split()
+ if len(fields) != 1:
+ self.fail("Invalid format for #" + token + " line: should be exactly one argument!")
+ symbol = fields[0]
+ condition = 'defined(' + symbol + ')'
+ if token == 'ifndef':
+ condition = '!' + condition
+
+ self.stack.append(("if", condition))
+ if self.verbose:
+ print(self.status())
+ return
+
+ previous_token, previous_condition = pop_stack()
+
+ if token == 'else':
+ self.stack.append(('else', negate(previous_condition)))
+ elif token == 'endif':
+ pass
+ if self.verbose:
+ print(self.status())
+
+if __name__ == '__main__':
+ for filename in sys.argv[1:]:
+ with open(filename, "rt") as f:
+ cpp = Monitor(filename, verbose=True)
+ print()
+ print(filename)
+ for line_number, line in enumerate(f.read().split('\n'), 1):
+ cpp.writeline(line)
projects / python / commitdiff