Peter Johnson [Tue, 20 Jun 2006 06:23:07 +0000 (06:23 -0000)]
In preparation for further work on optimizing multiple, break multiple apart
from bytecode.len into bytecode.mult_int. Add new function
yasm_bc_next_offset() to handle the most common use case of bytecode.len,
figuring out the following bytecode's offset.
Peter Johnson [Wed, 14 Jun 2006 07:23:49 +0000 (07:23 -0000)]
Implement first cut at optimizer step 2. Limitations:
- Does not detect cycles, so will infinite loop.
- Does not yet handle secondary expansion of bc offset bytecodes.
However, dwarf64_leb128 now generates essentially identical code to GAS
(except GAS generates relocs for in-file function refs, whereas yasm does not)
x86id tests also now work, but have not been checked yet (so no checkin of
new golden results yet).
Peter Johnson [Tue, 13 Jun 2006 06:37:37 +0000 (06:37 -0000)]
Build up interval tree. Change how SPECIAL_BC_OFFSET is handled for better
code commonality. Change optd.spans to a doubly-linked list and delete
directly from it rather than deactivating.
Peter Johnson [Mon, 12 Jun 2006 07:53:49 +0000 (07:53 -0000)]
Move towards building interval tree by splitting dependent value into
sym-sym terms. Starting to get some code duplication that should get
refactored somehow, but the first priority is getting it working.
Peter Johnson [Sun, 11 Jun 2006 20:30:46 +0000 (20:30 -0000)]
x86_bc_insn_expand(): Don't require ea/imm to be abs to expand, and check
span ID.
dwarf32_testhd.hex: New optimizer downsizes .text by 4 bytes, update.
Peter Johnson [Sat, 10 Jun 2006 18:35:36 +0000 (18:35 -0000)]
x86_bc_insn_tobytes(): Fix the case where we get here with imm8 post-op
still enabled; we need to fix up the immlen to 8 (as it's maintained with
the wordsize version up to this point).
Peter Johnson [Sat, 10 Jun 2006 16:54:45 +0000 (16:54 -0000)]
yasm_object_optimize(): Fix error/warning handling by using the new method.
effaddr.asm: Disable now-unsupported construct of [reg*(label2-label)].
x86_bc_insn_calc_len(): Don't destroy NULL intnum.
Peter Johnson [Sat, 10 Jun 2006 08:00:14 +0000 (08:00 -0000)]
Fix all tests with miscomparing object files. Dramatically clean up some
x86 code (mainly x86_checkea_calc_displen(), and parts of
x86_bc_insn_calc_len()) to use new yasm_value_get_intnum().
Allow NULL to be passed as bc to yasm_value_get_intnum() to prevent
calculation of PC-relative values (returning NULL instead).
Peter Johnson [Wed, 7 Jun 2006 04:01:33 +0000 (04:01 -0000)]
Get rid of calc_bc_dist_func, the only times this was used was when
yasm_common_calc_bc_dist was used. Rename yasm_common_calc_bc_dist to
yasm_calc_bc_dist, call it directly from the expr functions, and change
higher-level callers to boolean flags of whether to calculate bc distance
or not.
Peter Johnson [Wed, 31 May 2006 06:13:01 +0000 (06:13 -0000)]
Merge [1333]-[1543] (inclusive) into new-optimizer branch. This results in
a temporary regression of functionality but will yield some benefits later
on (chief among them is easier eventual merging back to the mainline!).
Peter Johnson [Mon, 22 May 2006 06:19:04 +0000 (06:19 -0000)]
* x86parse.gap: Update pushf generation to match popf for legalization and
prefix regarding pushfq vs 64-bit mode.
* pushf.asm, pushf-err.asm: New tests for pushf and popf in all modes.
Peter Johnson [Fri, 12 May 2006 02:12:19 +0000 (02:12 -0000)]
* x86id.c: Add support for 64-bit far jmp/call. These are not listed in the
AMD64 ISA reference but are in the Intel EM64T reference.
* x86arch.h: Add CPU_EM64T flag.
* x86parse.gap: Add CPU_EM64T as feature and enable it as part of "prescott".
* far64.asm: Test for 64-bit far jmp/call.
Peter Johnson [Thu, 11 May 2006 07:00:33 +0000 (07:00 -0000)]
Fix major bug related to multiple handling. Unfortunately this slows down
bytecode multiple output significantly, but this way handles generating
relocations correctly (otherwise extern foo; times 5 dd foo doesn't work!).
* bytecode.h (yasm_bc_tobytes): Remove multiple output parameter.
(yasm_bc_get_multiple): New.
* bytecode.c (yasm_bc_tobytes): Update to iterate through multiple here.
(yasm_bc_get_multiple): New.
* xdf-objfmt.c, elf-objfmt.c, bin-objfmt.c, coff-objfmt.c: Don't iterate
through multiple here.
* nasm-listfmt.c: Use yasm_bc_get_multiple() to get multiple instead of
yasm_bc_tobytes().
* bytecode.pxi: Update.
* tests/win32-relocovfl.asm: Tests both this and [1534] but generates too
huge of a file (3MB) to be put into the automated tests.
Peter Johnson [Wed, 10 May 2006 04:18:23 +0000 (04:18 -0000)]
* bc-int.h (yasm_bytecode_callback): Add reserve flag to indicate if the
bytecode outputs just space and not data.
* bytecode.pxi: Update to match.
* bytecode.c (yasm_bc_tobytes): Use reserve flag instead of checking
against bytecode_reserve as special case.
* dwarf2-info.c, dwarf2-line.c, dwarf2-dbgfmt.c, cv-symline.c, cv-type.c,
* stabs-dbgfmt.c, lc3bbc.c, x86bc.c: Update to set reserve=0.
* bc-align.c, bc-insn.c, bc-reserve.c, bc-org.c, bc-incbin.c, bc-data.c,
* bytecode.c: Split out bytecodes into separate files. While we're here,
merge LEB128 into general data bytecode.
Peter Johnson [Mon, 8 May 2006 08:08:15 +0000 (08:08 -0000)]
Try to combine as much data as possible into single raw (used to be string)
dataval data bytecode blocks. This drastically cuts down on dataval
overhead for large sets of constant data (savings of 30% or more on peak
memory usage in some cases). We may try to improve this even more down the
road by doing things like combining across multiple data bytecodes and
converting constant instructions into raw bytes.
* bytecode.c (yasm_bc_create_data): Add (optional) yasm_arch operand so that
bytes can be generated from constant values of >1 byte size. Do as much
simplification to constant raw bytes here as possible.
(bytecode_data): No longer a need to remember append_zero.
(bc_data_resolve, bc_data_output): Update to not handle append_zero.
(yasm_dataval): Rename string to raw.
(yasm_dv_create_string): Rename to:
(yasm_dv_create_raw): Here.
* bytecode.h (yasm_dv_create_string): Change into macro that casts parameters
into yasm_dv_create_raw().
(yasm_dv_create_raw): New.
modules: Update for new yasm_bc_create_data() parameter.
Peter Johnson [Mon, 8 May 2006 07:51:23 +0000 (07:51 -0000)]
* expr.c: Don't hit the allocator so hard with tiny temporary requests when
creating expressions; instead, keep a pool of 31 yasm_expr__items and pull
from those instead of malloc/free'ing.
Michael Urman [Sun, 7 May 2006 03:22:03 +0000 (03:22 -0000)]
Raise SymbolTable redefinition errors in Python.
Fix a double-free problem encountered during this testing.
Redefine symtab.define_special() vis parameter to match symrec.visibility.
Peter Johnson [Sun, 7 May 2006 02:35:50 +0000 (02:35 -0000)]
* errwarn.c (yasm_error_matches): Make real function, fix a bug with checking
against YASM_ERROR_GENERAL errors.
* errwarn.h (yasm_error_matches): Remove #defined function.
Peter Johnson [Thu, 4 May 2006 07:55:32 +0000 (07:55 -0000)]
Revamp error/warning handling, using a model similar to Python's internal
exception handling. There are now two layers an error or warning goes
through before it hits the user: first an error is logged via
yasm_error_set() (or yasm_warn_set() for a warning). Only one error may
be set, whereas multiple warnings can be set (yasm_warn_set maintains a
linked list). Then, calling yasm_errwarn_propagate() propagates any error
and/or warning(s) to an errwarns structure and associates the
errors/warnings with a line number at that time; this call also clears the
pending errors/warnings and allows new ones to be set. The propagate
function can safely be called when there are no pending error/warnings.
In addition, there are some helper errwarn functions that allow clearing of
an error/warning without propagating, getting it separately, etc.
Still yet to be done: changing most/all uses of yasm_internal_error() into
yasm_error_set(YASM_ERROR_ASSERTION).
The main advantage this change has is making libyasm functions feel much
more library like, and separating the user code line numbers from the inner
function error handling (e.g. intnum create functions only needed the line
number to trigger errors; this is no longer required).
The set/propagate/etc functions use global data structures to avoid passing
around a pointer to every function. This would need to be made thread-local
data in a threaded app. Errwarns containers (that keep associated line
numbers) are no longer global, so multiple source streams can be processed
separately with no conflict (at least if there's only a single thread of
execution).
Peter Johnson [Thu, 4 May 2006 03:59:39 +0000 (03:59 -0000)]
* ax_create_stdint_h.m4: Fix a couple bugs.
The m4_ifvaln() call doesn't seem to serve a purpose and generated bad shell
script code (trying to actually run a header file).
_STDINT_HEADER_INTPTR ifdef shouldn't wrap INTMAX_T check.
Peter Johnson [Sun, 30 Apr 2006 05:50:43 +0000 (05:50 -0000)]
* elf.c (elf_strtab_enty_set_str): Update following strtab indices since
string length can change. This was breaking symbol name lookups when the
filename was changed (e.g. for dwarf2).
* dwarf32_testhd.hex, dwarf64_leb128.hex: Fix testcases so they're correct.
Peter Johnson [Thu, 27 Apr 2006 03:48:57 +0000 (03:48 -0000)]
* nasm-bison.y (nasm_parser_directive): The objfmt directive must be the last
directive called; thus reorder dbgfmt_directive and objfmt_directive calls.
Peter Johnson [Wed, 26 Apr 2006 06:35:46 +0000 (06:35 -0000)]
* x86id.c (yasm_x86__finalize_insn): Don't check for explicit 64-bit
effective address size in GAS mode. Doing this breaks movabs.
* gas-movabs.asm: Test this.
Peter Johnson [Tue, 25 Apr 2006 03:42:45 +0000 (03:42 -0000)]
* nasm-bison.y: Enable dbgfmt directives from NASM parser. This allows e.g.
loc and file DWARF2 directives to be accessed using [loc ...] and [file ...]
just like .loc and .file work from the GAS parser.
Peter Johnson [Mon, 24 Apr 2006 05:53:26 +0000 (05:53 -0000)]
* nasm-token.re: Actually handle $-prefixed names correctly, stripping the $ prefix and allowing
any combination of characters following.
* win64-dataref.asm: Update to match (has a $-prefixed symbol).
Peter Johnson [Sat, 22 Apr 2006 04:08:00 +0000 (04:08 -0000)]
* nasm-preproc.c (nasm_preproc_input): Don't generate %line except at the
beginning of a line. The old way could insert a %line in the middle of very
long lines (e.g. >8192 chars).
* longline.asm: Test with an ~8500 character line.
Peter Johnson [Mon, 17 Apr 2006 03:53:04 +0000 (03:53 -0000)]
* python_test.sh: Set srcdir and PYTHON to reasonable defaults if not already
set, this allows easy manual execution of this script outside of make check.
Peter Johnson [Mon, 17 Apr 2006 03:21:41 +0000 (03:21 -0000)]
* python_test.sh: Use PYTHON variable to find python executable.
* python-yasm/tests/Makefile.inc: Pass PYTHON setting down through
TESTS_ENVIRONMENT.
* Makefile.am: Initialize TESTS_ENVIRONMENT to empty.
Peter Johnson [Sun, 16 Apr 2006 08:41:41 +0000 (08:41 -0000)]
* symrec.pxi: Revamp to more correctly generate Symbol objects, support
most dictionary functions (but not yet iterator protocol).
* yasm.pyx: Add Py_INCREF, Py_DECREF, malloc, free externals.
* coretype.pxi: Rename c_print to print_, add some missing yasm_expr_op
enum values.
Peter Johnson [Sat, 15 Apr 2006 08:03:42 +0000 (08:03 -0000)]
* python-yasm/Makefile.inc: Since we aren't using buildtools to call Pyrex,
we shouldn't need to remove the build directory. Not removing it speeds up
the build considerably (we don't have to rebuild all of the Python bindings
just because the Makefile changed, for example).
Peter Johnson [Sat, 15 Apr 2006 04:06:44 +0000 (04:06 -0000)]
Change the NASM preprocessor to use yasm_intnum and yasm_expr. This
decreases bloat and more importantly makes it possible to use >32-bit values
in the preprocessor.
This has NOT been heavily tested, so there may easily be bugs. I've not yet
decided whether to merge this to 0.5.0 final for this reason.
Inspired by: Jason Chen <jchen@centtech.com> patches to use "long long"
rather than "long" in the preprocessor to enable >32-bit values.
* coretype.h (yasm_op): Add XNOR, LXOR, LXNOR, LNOR.
* intnum.c (yasm_intnum_calc): Calculate the above.
* expr.c (expr_is_constant, expr_can_destroy_int_left)
(expr_can_destroy_int_right, ...): Actually handle logical operations,
including the new ones.
* intnum.c (yasm_intnum_get_str): New; gets a signed decimal string
representation of an intnum.
* intnum.h (yasm_intnum_get_str): Prototype.
* nasm-preproc.c (nasm_preproc_input): Close a memory leak.
* nasm.h: Clean out a lot of cruft we don't use.
(tokenval): Change t_integer and t_inttwo to yasm_intnums.
(evalfunc): Remove fwref and hints parameters (cleanup), change return value
to yasm_expr.
* nasm-eval.c: Massively rewrite to just call appropriate yasm_expr creation
functions rather than calculating the value here. Overall recursive descent
parsing structure is unchanged.
* nasmlib.c: Remove a lot of now-unused functions.
(nasm_readnum): Use yasm_intnum functions and return that.
(nasm_readstrnum): Likewise.
* nasmlib.h: Update prototypes.
* nasm-pp.c: Change to use intnum/expr as necessary.
* nasm-preproc.c (nasm_preproc_destroy): Don't call nasm_eval_cleanup,
it's been deleted.
* nasmpp-bigint.asm: New test for >32-bit preproc values.
* ifcritical-err.errwarn: The new code doesn't generate a duplicate warning.
Peter Johnson [Wed, 12 Apr 2006 04:06:44 +0000 (04:06 -0000)]
Correctly handle input characters >127 by using unsigned char in the re2c
tokenizers. Signed chars >127 are negative, and thus aren't caught by the
[\000-\377] range.
* gas-parser.h (YYCTYPE): Change to unsigned char.
* gas-bison.y, gas-token.re: Cast as necessary to char.
* nasm-parser.h, nasm-bison.y, nasm-token.re: Likewise.
* lc3bid.re: Likewise.
Peter Johnson [Sun, 9 Apr 2006 06:05:37 +0000 (06:05 -0000)]
* ax_create_stdint_h.m4: Don't bother using head -1 to get the first line of
gcc --version; this is just extra information. POSIX needs -n 1 instead of
-1; instead of dealing with this, just disable this function.
Peter Johnson [Thu, 6 Apr 2006 05:55:41 +0000 (05:55 -0000)]
Take #2 on reversioning: decouple version and build from autoconf version
(which is used for the tar.gz name). Also clean up Mkfiles/config.h
* configure.ac: Change autoconf version back to HEAD, add new PACKAGE_INTVER
and PACKAGE_BUILD config.h defines.
* cv-symline.c, yasm.c: Use PACKAGE_INTVER and PACKAGE_BUILD instead of
PACKAGE_STRING.
* genversion.c: Likewise.
* Mkfiles: Clean up and add PACKAGE_INTVER and PACKAGE_BUILD.
Peter Johnson [Wed, 5 Apr 2006 07:14:41 +0000 (07:14 -0000)]
Implement better versioning. From now on, trunk's version will be the
current major and minor version but with subminor version = 99. All releases
and snapshots will have a build version of the current svn version. Version
information is put into predefined macros in the NASM preproc:
__YASM_MAJOR__, __YASM_MINOR__, __YASM_SUBMINOR__, __YASM_BUILD__,
__YASM_VERSION_ID__, __YASM_VER__.
__YASM_VER__ does not have the build version as part of the string, and
__YASM_VERSION_ID__ does not incorporate the build version.
If the build version is "HEAD" (or other non-numeric), __YASM_BUILD__ is
set to 0.
* configure.ac: Set version to 0.4.99.HEAD for trunk.
* genversion.c: Generate version.mac for the NASM preproc version macros.
* Makefile.inc: Hook into build.
Peter Johnson [Wed, 5 Apr 2006 05:39:23 +0000 (05:39 -0000)]
* COPYING: Update verbiage, add list of contributors and copyright notice.
* genstring.c: Generate string array from source file.
* Makefile.am: Build genstring.
* Makefile.inc: Use genstring to create license.c from COPYING.
* yasm.c: Include license.c and use it for new --license option. Shorten
--version option to just displaying version, compile date, and very short
copyright message.
Peter Johnson [Wed, 5 Apr 2006 02:26:19 +0000 (02:26 -0000)]
Quiet warnings in Pyrex code by adding "-w" if GCC detected.
* configure.ac: Pass GCC value to Makefile
* Makefile.inc: Pass GCC value to python-setup.txt
* setup.py: Check GCC value and append -w if necessary.
Peter Johnson [Tue, 4 Apr 2006 07:51:42 +0000 (07:51 -0000)]
Be much smarter at checking for and running Pyrex. Pyrex is a Python module
and the "pyrexc" command is just a wrapper, so instead of looking for
"pyrexc" look for the module instead.
* pyrex.m4: New macro to check Pyrex version.
* m4/Makefile.inc: Add to dist.
* configure.ac: Use macro to check for Pyrex >= 0.9.3.
* python-yasm/Makefile.inc: Run Python directly instead of pyrexc wrapper.
Peter Johnson [Tue, 4 Apr 2006 06:07:32 +0000 (06:07 -0000)]
Hook Python module into the build (even though it's pretty incomplete),
enabled with --enable-python (defaults to auto-detect).
Thanks to some magic in python-yasm/Makefile.inc and setup.py, this actually
plays nicely with automake distcheck.
* m4/pythonhead.m4: Script to find Python.h.
* m4/Makefile.inc: Include it in distfiles.
* configure.ac: Add --enable-python option and Python and Pyrex detection.
* tools/Makefile.inc: Pull tools/python-yasm/Makefile.inc into build.
* python-yasm/Makefile.inc: Add automake build magic for extension.
* setup.py: Likewise.
projects / yasm / log