diff --git a/.gitignore b/.gitignore index 42e0dff..94a6fae 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,9 @@ *pyc *~ +doc/WIPNOTES/0.RandomDevNotes.md parser.out parsetab.py test-data/genprogs -site/site/*html \ No newline at end of file +site/site/*html +contrib/ diff --git a/CHANGELOG b/CHANGELOG index a14757e..01dd5c1 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -3,7 +3,97 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/), within reason. (Prior to a 1.0.0 release minor version increases may be backwards -incompatible) +incompatible) See doc/Versioning.md for more details + +## [0.1.26] - UNRELEASED + +### New + +* + +### What's been fixed? + +* + +### Internal Changes + +* + +### Other + +* + +## [0.1.25] - 2018-02-03 + +The focus of this release is the result of the independent intermediate +node refactoring project I was undertaking. This was to transform the list +oriented data structure for representing a "pure" intermediate form of +code into a more stuctured/generalisable format. + +The motivation behind this change was the realisation that implementation +of end user functions (ie `def`) would be very difficult with this more +structured approach. + +### New + +* `doc/Versioning.md` - semantic versioning as it applies to pyxie +* `doc/WIPNOTES/6.Models.md` - start of some docs around the models in use +* Added explicit notes on licensing of pyxie's output. + (Short version: I view the output as being derived from your code by you) +* Language focussed examples / acceptance tests added: + `if` `if-else` `pass` `print` `while-break` `while-continue` +* Change `arduino` profile to support the `Adafruit_NeoPixel` library +* `neopixel` example added + +### What's been fixed? / Improved + +* Handling if if/elif/else improved/fixed +* added `clean.sh` to `arduino` example +* added `clean.sh` to `servo` example +* added `README.md` to `simplest_function` example (won't work yet) + +### Internal Changes + +* `bin/pyxie` now pulls in functionality from `pyxie.api` to be clearer + about what the API is +* added `pyxie/api.py` - public API for embedding pyxie. (Assumes + ubuntu host) Core contents: + * `initialise_API(profile="default")` - Call first + * `set_profile(profile_name)` - Call later + * `PyxieAPI.parse(filename)` - parse file, output goes to console + * `PyxieAPI.analyse(filename)` - parse & analyse flle, output to console + * `PyxieAPI.codegen(filename, result_filename=None)` - parse through + code generation. Output to console + * `PyxieAPI.compile(filename, result_filename=None)` - Compile file, + result ends up as "result_filename" +* `pyxie/core` - changed over to drive the conversion of pynodes to + cppnodes via iinodes. (aim is to simplify pynodes down the line and + make iinodes our main location for analysis/etc) +* Minor changes to `.gitignore` +* Minor change to the `Makefile` to make editting simpler for me... +* Update `clib.py` based on changes to `clib` +* `pyxie/codegen/simple_cpp.py` - functionality shifted out to + `pyxie/models/cppnodes.py` to allow a model driven approach +* `pyxie/model/cppnodes.py` Created. A better code representation + model for C++. Code generation therefore means transforming from the + `iiNode`s to `CppNode`s +* `pyxie/model/iinodes.py` - Introduces `iiNode`s - which are used to + represent an intermediate version of a program such that it can bridge + the conversion between languages +* `pyxie/model/pynodes/operators.py` - added `args()` method to a number + of classes to unify behaviours +* `pyxie/model/transform.py` - Major change - converts `PyNode`s to + `iiNode` representation rather than json objects. + +### Other + +* `doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md` corrected release date +* `doc/newsletter/XX-TBD.Pyxie-2017-Goals.md` - Unreleased newsletter. + Interesting notes, will get reused if appropriate later. Rather moot + contents now though. +* `doc/newsletter/08-TBD.Focus-Or-Pyxie-Release-X.X.XX.md` Template for + next newsletter + ## [0.1.24] - 2016-11-10 @@ -66,7 +156,6 @@ incompatible) * Add special case for function calls like print #37 - ## [0.1.23] - 2016-10-11 Not backwards compatible due to changes to "print" diff --git a/COPYING.output.md b/COPYING.output.md new file mode 100644 index 0000000..97bb4aa --- /dev/null +++ b/COPYING.output.md @@ -0,0 +1,290 @@ +License regarding output code +============================= + +Really short version +==================== + +The output from pyxie - is for yours to do with as you like. +This document tries to make sure this unamiguous. + +Longer version +============== + +While musing random stuff, things like Bison sprang to mind. Bison is a +GPL'd parser generator, and placed restrctions over its output. This +led to all sorts of questions over time, and so on. In the case of Pyxie, +I want to avoid those questions. They will frusrate me, and frustrate you +for zero benefit. + +So, to be clear, I view the output of pyxie to be yours. You wrote the +source that Pyxie transforms, and the transformed code is a derivative +of your code. Therefore the output from pyxie is your code, generally. + +As a result, in files /directly/ derived from your code the following +statement is included at the top: + + // + // This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie + // + // This file is derived from the user's source code, copyright resides + // with the author of the user's source code. + // + // You can edit this header in any way suitable for your project + // + +There will always be elements though that are actually parts of pyxie that +get copied into your code as part of that transformation. An example would +include things like how python iterators and generators are implemented. + +Clearly I'm not going to assign copyright over those bits of code to you. +Therefore I need to give you a license for those. + +As a result, in files that are essentially derived from Pyxie, the following +statement is included at the top: + + // + // This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie + // + // This file contains parts of code derived from Pyxie itself. It may be used + // under the following terms: + // + // Permission is hereby granted, free of charge, to any person obtaining + // a copy of this software and associated documentation files (the "Software"), + // to deal in the Software without restriction, including without limitation + // the rights to use, copy, modify, merge, publish, distribute, sublicense, + // and/or sell copies of the Software, and to permit persons to whom the Software + // is furnished to do so, subject to the following conditions: + // + // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + // SOFTWARE. + // + // For more information see: http://www.sparkslabs.com/pyxie/COPYING.output.md + // + +Note: this is a more permissive version of the MIT license. It's more permissive +because it actually allows you to remove the notice. If you're distributing your +source to others though, this probably wouldn't be a good idea. + + +Worked example +============== + +Pyxie is in essence a code generator. It takes a definition - such as something like this: + + #include + + pin = 6 + number_of_pixels = 16 + delayval = 500 + + pixels = Adafruit_NeoPixel(number_of_pixels, pin, NEO_GRB + NEO_KHZ800) + + pixels.begin() + + while True: + for i in range(number_of_pixels): + pixels.setPixelColor(i, pixels.Color(0,150,0)) + pixels.show() + delay(delayval) + +And generates a main file like this: + + // + // This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie + // + // This file is derived from the user's source code, copyright resides + // with the author of the user's source code. + // + // You can edit this header in any way suitable for your project + // + + #include + + #include "iterators.cpp" + + void setup() { + int delayval; + int i; + int number_of_pixels; + int pin; + Adafruit_NeoPixel pixels; + range range_iter_1; + + pin = 6; + number_of_pixels = 16; + delayval = 500; + pixels = Adafruit_NeoPixel(number_of_pixels, pin, (NEO_GRB + NEO_KHZ800)); + (pixels).begin(); + while (true) { + + range_iter_1 = range(number_of_pixels); + while (true) { + i = range_iter_1.next(); + if (range_iter_1.completed()) + break; + + + (pixels).setPixelColor(i, (pixels).Color(0, 150, 0)); + (pixels).show(); + delay(delayval); // Itself uses i + } + ; + }; + } + + void loop() { + } + +You'll note though that the output code includes code from Pyxie itself - iterators.cpp +and iterators.hpp. These aren't written by you and need a license. As a result the +output of these looks like this: + +iterators.cpp: + + // + // This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie + // + // This file contains parts of code derived from Pyxie itself. It may be used + // under the following terms: + // + // Permission is hereby granted, free of charge, to any person obtaining + // a copy of this software and associated documentation files (the "Software"), + // to deal in the Software without restriction, including without limitation + // the rights to use, copy, modify, merge, publish, distribute, sublicense, + // and/or sell copies of the Software, and to permit persons to whom the Software + // is furnished to do so, subject to the following conditions: + // + // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + // SOFTWARE. + // + // For more information see: http://www.sparkslabs.com/pyxie/COPYING.output.md + // + + #include "iterators.hpp" + + struct range : public Generator { + int start; + int end; + int step; + + int index; + + range() : start(0), end(0), step(1), index(0) { }; + range(int end) : start(0), end(end), step(1), index(0) { }; + range(int start, int end) : start(start), end(end), step(1), index(start) { }; + range(int start, int end, int stepsize): start(start), end(end), step(stepsize), index(start) { }; + ~range() { }; + + virtual int next() { + GENERATOR_START + + while ( step>0 ? index < end : index > end) { + YIELD(index); + index = index + step; + } + + GENERATOR_END + } + }; + +iterators.hpp looks like this: + + #ifndef PYXIE_ITERATORS_HPP + #define PYXIE_ITERATORS_HPP + + // + // This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie + // + // This file contains parts of code derived from Pyxie itself. It may be used + // under the following terms: + // + // Permission is hereby granted, free of charge, to any person obtaining + // a copy of this software and associated documentation files (the "Software"), + // to deal in the Software without restriction, including without limitation + // the rights to use, copy, modify, merge, publish, distribute, sublicense, + // and/or sell copies of the Software, and to permit persons to whom the Software + // is furnished to do so, subject to the following conditions: + // + // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + // SOFTWARE. + // + // For more information see: http://www.sparkslabs.com/pyxie/COPYING.output.md + // + + /* + * Python Style Iterators in C++, based in part on experiment work in Kamaelia + * Redone to remove use of generators, so that this can be used on Arduino + * + */ + + #define GENERATOR_START if (this->__generator_state == -1) { return __default_value; } switch(this->__generator_state) { default: + #define YIELD(value) this->__generator_state = __LINE__; return ((value) ); case __LINE__: + #define GENERATOR_END }; this->__generator_state = -1; return __default_value; + + template + struct Iterator { + virtual T next()=0; + virtual bool completed()=0; + }; + + template + class Generator : public Iterator { + protected: + int __generator_state; + public: + T __default_value; + Generator() : __default_value(T()) { }; + ~Generator() { }; + virtual bool completed() { return __generator_state==-1; }; + }; + + #endif + + +Why? +==== + +In Pyxie I make pains to try and make the output code readable (within reason). +The reason for this is simple you may decide to stop using pyxie if you hit the +limits of the language and to continue to proceed using the output code. The +reason for this is because I'm a pragmatist :-) + +In that situation you don't want to be in the situation where you suddenly can't +use the code in ways that you want to. I wouldn't want that for me, so it wouldn't +be right for me to expect that from you. Hence this explicit licensing over +output files. + + +Open Source +=========== + +All that said, consider this: + +* Pyxie is open source so, if possible, consider leaving code you create + using pyxie open source, + +* If you can't and you make a profit in anyway, consider making a donation + to the pyxie project, to help run servers, boost morale etc. + +Failing that, a christmas (or similar) card would be nice - but of course none +of this is obligatory :) + +Enjoy :-) + + +Michael, December 2016 diff --git a/MANIFEST.in b/MANIFEST.in index 18ed0c4..046f577 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -2,8 +2,12 @@ recursive-include examples * recursive-include features * recursive-include test-data/progs * recursive-include bin * +recursive-include clib * recursive-include doc * +recursive-include site * include README.md include stdeb.cfg - +include COPYING +include COPYING.output.md +include README.rst diff --git a/Makefile b/Makefile index 1d8d886..48a46e6 100644 --- a/Makefile +++ b/Makefile @@ -17,7 +17,7 @@ PYTHON=`which python` DESTDIR=/ PROJECT=pyxie -VERSION=0.1.24 +VERSION=0.1.26 all: @echo "make source - Create source package" @@ -32,6 +32,7 @@ all: @echo "make test - run behave and other tests" @echo "make clean - Get rid of scratch and byte files" @echo "make test - Run any unit tests" + @echo "make edit - convenience to launch editor for dev" source: $(PYTHON) setup.py sdist $(COMPILE) @@ -49,10 +50,10 @@ ppadeb: @echo "Clean up dist before uploading to pypi, or it'll contain too much junk" use: - sudo dpkg -i dist/deb_dist/python-$(PROJECT)*deb + sudo dpkg -i dist/deb_dist/python3-$(PROJECT)*deb purge: - sudo apt-get purge python-$(PROJECT) + sudo apt-get purge python3-$(PROJECT) clean: $(PYTHON) setup.py clean @@ -73,3 +74,6 @@ devloop: purge distclean deb use test: PYTHONPATH="." ./bin/pyxie --test run-tests # behave + +edit: + ((kate --new -s Pyxie /dev/null 2>/dev/null)&)& diff --git a/README.rst b/README.rst index 3ef15e2..6857ee9 100644 --- a/README.rst +++ b/README.rst @@ -4,8 +4,8 @@ Pyxie -- A Little Python to C++ Compiler What job does / will this do? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The aim of this project is to allow a adults and children to write code -in a familiar high level language that can then be compiled to run on an +The aim of this project is to allow adults and children to write code in +a familiar high level language that can then be compiled to run on an arbitrary embedded system - that is devices with very low power CPUs and very little memory. (ie devices too small to host a python interpreter/runtime) @@ -240,6 +240,9 @@ Release History Release History: +- 0.1.26 - UNRELEASED - TBD +- 0.1.25 - 2018-02-03 - Major changes to internal representation and + some new examples - 0.1.24 - 2016-11-10 - Fix assignment in else/elif, update to support Ubuntu 16.04LTS, initial steps on function/def statement support - 0.1.23 - 2016-10-11 - Major arduino profile improvements, @@ -431,4 +434,4 @@ As of 0.1.23, the print\_statement has been removed. As well as being simplifying the syntax, it also means that Arudino statements like Serial.print now become legal statements. -Michael Sparks, November 2016 +Michael Sparks, February 2018 diff --git a/bin/pyxie b/bin/pyxie index 33334e6..ffda5e0 100755 --- a/bin/pyxie +++ b/bin/pyxie @@ -18,129 +18,32 @@ from __future__ import print_function from __future__ import absolute_import -import pprint import sys -from pyxie.core import parse_file -from pyxie.core import analyse_file -from pyxie.core import compile_file -from pyxie.core import codegen_phase -from pyxie.core import compilation_tests -from pyxie.core import compile_testfile -from pyxie.core import parsing_tests -from pyxie.core import parse_testfile +from pyxie.api import CommandLineDispatcher +from pyxie.api import TestsLauncher +from pyxie.api import StandardOptions +from pyxie.api import PyxieAPI -from pyxie.core import testprogs_dir +from pyxie.api import initialise_API -import pyxie.parsing.context - -# This is to support compilation profiles. This may or may not turn out to -# be a good approach. (#3/#3.x) -profile = "default" class BadArguments(Exception): pass -# The purpose of this command line dispatcher is to simplify options handling at the -# expense of being a bit pickier about it. - -class CommandLineDispatcher(object): - @classmethod - def handles(klass, command): - return command.replace("-","_") in [ x for x in dir(klass) if not(x.startswith("_"))] - # - @classmethod - def handle(klass, command, *args): - f = getattr(klass, command.replace("-","_")) - f(*args) - -class TestsLauncher(CommandLineDispatcher): - @staticmethod - def run_tests(): - """Run all tests""" - parsing_tests() - compilation_tests(profile) - - @staticmethod - def parse_tests(): - """Just run parse tests""" - parsing_tests() - - @staticmethod - def compile_tests(): - """Just run compile tests""" - compilation_tests(profile) - - @staticmethod - def parse(filename): - """Parses a given test given a certain filename""" - AST = parse_testfile(testprogs_dir, filename) - pprint.pprint(AST) - pprint.pprint(AST.__json__()) - - @staticmethod - def compile(filename): - """Parses a given test given a certain filename""" - compile_testfile(testprogs_dir, filename, profile) - - -class StandardOptions(CommandLineDispatcher): - @staticmethod - def parse(filename): - """parses the given filename, outputs result to console""" - AST = parse_file(filename) - pprint.pprint(AST) - pprint.pprint(AST.__json__()) - if 0: - import json - print("Also writing a copy to ", filename+".json") - f = open(filename+".json", "w") - f.write(json.dumps(AST.__json__())) - f.close() - - @staticmethod - def analyse(filename): - """parses and analyse the given filename, outputs result to console""" - analyse_file(filename) - - @staticmethod - def codegen(filename, result_filename=None): - """parses, analyse and generate code for the given filename, outputs result to console. Does not attempt compiling""" - codegen_phase(filename, result_filename, profile) - - @staticmethod - def compile(filename, result_filename=None): - """compiles the given file to path/to/filename -- result_filename can be provide an alternative output name""" - compile_file(filename, profile, result_filename) def main(argv): global profile if len(argv) < 2: raise BadArguments() - # Create a default profile - whether or not it is used. - print("CREATING A CONTEXT") - context = pyxie.parsing.context.Context() - context.tag = "PROFILE: (default/None)" + profile = "default" - # If a profile is pulled in/defined, use that if argv[1] == "--profile" and len(argv)> 2: profile = argv[2] - argv[1:3] = [] - - print("USING PROFILE: ", profile) - context.tag = "PROFILE:"+profile + argv[1:3] = [] # Chop out arguments relating to profile - if profile == "arduino": # TODO: Delete me, this is a test - from pyxie.profile.arduino import initialise_profile - initialise_profile(context) - else: - print("UNKNOWN PROFILE") - print("USING PROFILE: ", profile) - context.tag = "PROFILE:"+profile - - print("CREATED", context) - pyxie.parsing.context.profile_context = context + initialise_API(profile_name=profile) if argv[1] == "--test": command = argv[2] @@ -155,6 +58,7 @@ def main(argv): # Not handled by anything raise BadArguments("Bad Arguments") + def show_help(): print("""\npyxie -- A little python compiler\nUsage:\n pyxie -- show runtime arguments @@ -182,4 +86,4 @@ if __name__ == "__main__": print("USAGE ERROR:", e) print print("Original traceback follows") - raise(e) + raise diff --git a/clib/iterators.cpp b/clib/iterators.cpp index 7860874..2257c7f 100644 --- a/clib/iterators.cpp +++ b/clib/iterators.cpp @@ -1,3 +1,27 @@ +// +// This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie +// +// This file contains parts of code derived from Pyxie itself. It may be used +// under the following terms: +// +// Permission is hereby granted, free of charge, to any person obtaining +// a copy of this software and associated documentation files (the "Software"), +// to deal in the Software without restriction, including without limitation +// the rights to use, copy, modify, merge, publish, distribute, sublicense, +// and/or sell copies of the Software, and to permit persons to whom the Software +// is furnished to do so, subject to the following conditions: +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. +// +// For more information see: http://www.sparkslabs.com/pyxie/COPYING.output.md +// + #include "iterators.hpp" struct range : public Generator { diff --git a/clib/iterators.hpp b/clib/iterators.hpp index bc369b7..ac894e1 100644 --- a/clib/iterators.hpp +++ b/clib/iterators.hpp @@ -1,3 +1,27 @@ +// +// This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie +// +// This file contains parts of code derived from Pyxie itself. It may be used +// under the following terms: +// +// Permission is hereby granted, free of charge, to any person obtaining +// a copy of this software and associated documentation files (the "Software"), +// to deal in the Software without restriction, including without limitation +// the rights to use, copy, modify, merge, publish, distribute, sublicense, +// and/or sell copies of the Software, and to permit persons to whom the Software +// is furnished to do so, subject to the following conditions: +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. +// +// For more information see: http://www.sparkslabs.com/pyxie/COPYING.output.md +// + #ifndef PYXIE_ITERATORS_HPP #define PYXIE_ITERATORS_HPP diff --git a/doc/0.1.25-ReleaseNotes.md b/doc/0.1.25-ReleaseNotes.md new file mode 100644 index 0000000..ae4c589 --- /dev/null +++ b/doc/0.1.25-ReleaseNotes.md @@ -0,0 +1,142 @@ +# Release 0.1.25 - Major changes to internal representation and some new examples + +## Overview + +### Summary + +The focus of this release is the result of the independent intermediate +node refactoring project I was undertaking. This was to transform the list +oriented data structure for representing a "pure" intermediate form of +code into a more stuctured/generalisable format. + +The motivation behind this change was the realisation that implementation +of end user functions (ie `def`) would be very difficult with this more +structured approach. + +### Rationale behind changes + +Before this change, pyxie used the following data structures, and code +phases when parsing, analysing, transforming and performing code +generation. + +These were not ideal: + +* Code phase: Parser - read in the source, parsed it, and created a + data structure + * Data structure: pynodes - these represent the concrete python + syntax, and are generated during the code generation phase. These + are python objects in a class hierarchy. Once created they are + analysed and placed into a context to analyse datatypes. +* Code phase: Analysis. - Analyses the code, and decorates the existing + pynodes with type information to understand the programs data and data + types. (does not create a new data structure). +* Transform phase: walks the pynode CST, and generates an intermediate + data structure intended to represent the program in the abstract + independent of the language used. An independent intermediate form + if you like. + * Data structure: independent intermediate form - This is used + to model the program in a "pure form" - which isn't constrained + by the source language, and contains enough information for the + target language output to be generated. That was the theory. In + practice it was a nested list of list of lists ... Not ideal. + More on this below. +* Code generation phase: This walked the independent intermediate form + (the lists of lists), and created an output data structure representing + the concrete final program. + - The third/final data structure is intended to represent the final + output language. ie it is intended to be a concrete representation of + the output language. This final data structure is then capable of + creating the output. Again, forms a hierarchy and could be called + "CppNodes" (Though they weren't before this change) + +The problem here is that while the pynodes are currently well defined (to +a large extent) and that the CppNodes are pretty well defined (even if +they could be better), the independent intermediate form sucked because +it was just nested lists. This meant in practice the code was fragile, +difficult to change, and increasingly difficult to work with. In the early +days of Pyxie this simplistic structure made sense. However as data +analysis becomes more complex and tricky. This mirrors the fact that +the same was true in the early days of the parsing side. + +So the focus of this sub-project was to replace the intermediate form, +and tighten up the differences and make clearer in the code base that +we have PyNodes, iiNodes, and CppNodes, where: + +* Pynodes - are the current structures, unchanged - note these are + currently prefixed Py - PyDefStatement for example. +* CppNodes - are map to objects/hierarchy/etc that represents C++ programs, + but made clearer (slightly). These are now prefixed Cpp rather than the + previous C_ which *some* of the objects use. +* iiNodes - represent the independent, intermediate nodes. Since iiNodes + need to be used in source code where there are either PyNodes + iiNodes + or CppNodes + iiNodes, they have the prefix "ii" enable differentiation. + +Since all 3 of these are actually *models*, the code for these has all +moved to sit below pyxie.models. + +At some point there will be a need to restructure the code more generally. +Parsing, transforming to IINodes and code generation are actually all +transforms, and should sit together. That was out of scope for this +already relatively major internal change. + +## Changelog + +### New + +* `doc/Versioning.md` - semantic versioning as it applies to pyxie +* `doc/WIPNOTES/6.Models.md` - start of some docs around the models in use +* Added explicit notes on licensing of pyxie's output. + (Short version: I view the output as being derived from your code by you) +* Language focussed examples / acceptance tests added: + `if` `if-else` `pass` `print` `while-break` `while-continue` +* Change `arduino` profile to support the `Adafruit_NeoPixel` library +* `neopixel` example added + +### What's been fixed? / Improved + +* Handling if if/elif/else improved/fixed +* added `clean.sh` to `arduino` example +* added `clean.sh` to `servo` example +* added `README.md` to `simplest_function` example (won't work yet) + +### Internal Changes + +* `bin/pyxie` now pulls in functionality from `pyxie.api` to be clearer + about what the API is +* added `pyxie/api.py` - public API for embedding pyxie. (Assumes + ubuntu host) Core contents: + * `initialise_API(profile="default")` - Call first + * `set_profile(profile_name)` - Call later + * `PyxieAPI.parse(filename)` - parse file, output goes to console + * `PyxieAPI.analyse(filename)` - parse & analyse flle, output to console + * `PyxieAPI.codegen(filename, result_filename=None)` - parse through + code generation. Output to console + * `PyxieAPI.compile(filename, result_filename=None)` - Compile file, + result ends up as "result_filename" +* `pyxie/core` - changed over to drive the conversion of pynodes to + cppnodes via iinodes. (aim is to simplify pynodes down the line and + make iinodes our main location for analysis/etc) +* Minor changes to `.gitignore` +* Minor change to the `Makefile` to make editting simpler for me... +* Update `clib.py` based on changes to `clib` +* `pyxie/codegen/simple_cpp.py` - functionality shifted out to + `pyxie/models/cppnodes.py` to allow a model driven approach +* `pyxie/model/cppnodes.py` Created. A better code representation + model for C++. Code generation therefore means transforming from the + `iiNode`s to `CppNode`s +* `pyxie/model/iinodes.py` - Introduces `iiNode`s - which are used to + represent an intermediate version of a program such that it can bridge + the conversion between languages +* `pyxie/model/pynodes/operators.py` - added `args()` method to a number + of classes to unify behaviours +* `pyxie/model/transform.py` - Major change - converts `PyNode`s to + `iiNode` representation rather than json objects. + +### Other + +* `doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md` corrected release date +* `doc/newsletter/XX-TBD.Pyxie-2017-Goals.md` - Unreleased newsletter. + Interesting notes, will get reused if appropriate later. Rather moot + contents now though. +* `doc/newsletter/08-TBD.Focus-Or-Pyxie-Release-X.X.XX.md` Template for + next newsletter diff --git a/doc/Versioning.md b/doc/Versioning.md new file mode 100644 index 0000000..403b792 --- /dev/null +++ b/doc/Versioning.md @@ -0,0 +1,270 @@ +--- +Summary: My take on semantic versioning and Pyxie +Updated: January 2017 +Started: November 2016 +Version: 0.1.24 +Author: Michael Sparks +--- +# What does Pyxie's Version Mean? + +Pyxie's public released version as of today is 0.1.24. This does actually +have some measure of meaning because it's based on semantic versioning +as defined at http://semver.org + + +## MAJOR.MINOR.RELEASE *not* MAJOR.MINOR.PATCH + +SEMVER normally uses the form MAJOR.MINOR.PATCH, I use MAJOR.MINOR.RELEASE + +The following is a sample release schedule used below. + +* 0.5.56 - last dev release before 1.x.x release +* 1.6.57 - First stable 1.x.x release +* 0.6.58 - First dev release after 1.6.57, continuing development, adding new features +* 0.6.59 - Next dev release adding new features +* 0.7.60 - Next dev release adding new features +* 0.7.61 - Next dev release fixing bugs +* 1.7.62 - Second stable 1.x.x release +* 0.7.63 - Next dev release fixing bugs +* 0.7.64 - Next dev release adding features +* 0.8.65 - Next dev release adding major functionality +* 0.9.65 - Next dev release adding more major functionality +* 0.9.66 - Next dev release - candidate stable release perhaps. +* 2.9.67 - This stable release - adds major functionality, but in a + backwards incompatible fashion, so MAJOR version bumps. + + +## Key principles + +Given: MAJOR.MINOR.RELEASE +For all releases: + +* **RELEASE** always increments over time with no branching. (This should + be visible above). It means the code has changed and there's a reason + for it! (features/bugs/docs/etc) + +* **MINOR** always increases. This is a slightly odd choice, but the intent + here is that MINOR is a measure of total project status/complexity. + +* **MAJOR** = 0 --> dev release - always remains zero + +* **MAJOR** > 0 --> non-dev release - only ever increases when there are + backwards incompatible releases. +* Dev and non-dev releases may follow each other as per the slightly + unusual release schedule described above. + +For non-dev releases: + +* No suprises when installing a release where MAJOR is unchanged. + +* Where MAJOR.MINOR are unchanged it should not affect a user negatively. + +* Installing a release where MINOR changes should contain a benefit + such as major bug fixes or extra features + +* Backwards incompatible changes must bump the MAJOR number. + +* The MAJOR number *may* change where there are significant benefits + over the previous first MAJOR release. (eg after many minor + releases perhaps) + +* Version numbers must be compatible with semantic versioning + +* The API the release relates to must be clearly defined: + - If it's not in the API docs, it isn't part of the release, + and is subject to change. + +* non-dev releases may be alpha or beta releases, and this is not + as yet encoded in the version number + +For dev releases: + +* If MAJOR.MINOR has changed, then there is new major or breaking + functionality. + +* Otherwise, while not required, it is *likely* compatible with a + previous version with the same MAJOR/MINOR number. Note that it + is a dev release, this is not required. + + +## How this applies to Pyxie + + +### Current meaning of 0.1.24 -- as it applies to pyxie + +Given 0.1.24 stands for MAJOR.MINOR.RELEASE, it means the following: + +* MAJOR - 0 - This means that I view Pyxie as still in it's initial + development phase. ie the code it compiles looks increasingly python + like - and is a valid subset of python, but doesn't really have the + language coverage you would expect of a 1.x.x release. + +* MINOR - 1 - If this was 0, it would mean that pretty much all bets are off + about internal structure, and external API. For this to bump to 1 it + means that Pyxie is actually useful now for *something*. In particular, + at present, you can use it right now for controlling simple quadruped + robots with sensors. + +* RELEASE - 24 - This is literally a release number. It means that this is + the 24'th release of the codebase. + + +### Current meaning of 0.MINOR.RELEASE + +Until I hit a 1.x.x release, these numbers will increase as follows: + +* MAJOR - 0 - This will be 0 until I hit a 1.x.x release. See the next + section for criterion I have for this. + +* MINOR - MINOR level versions are guaranteed to increase permanently and + not reset to zero. This will increment for one or both of the following + reasons: + - The language as implemented gains a major piece of practical + functionality + - The external API (command line) changes in an incompatible manner to the + previous release. (If things are simply *added* this will probably + not change) + +* RELEASE - This will be incremented for every release. The change may be + small or it may be large. PATCH level versions are guaranteed to increase + permanently and not reset to zero. + + +### Current meaning of 1.MINOR.RELEASE + +Eventually I will hit a 1.x.x release, at which point this section applies. + + +### 1.x.x criterion + +The criterion I have in mind follow. These may change. If so this will be +made clear below. Specifically I would expect (at minimum) the following +features to be implemented. + +Definitely: + +* Lists +* Dictionaries +* Strings +* Simple user functions +* Tested with arduino profile + +Maybe: + +* Simple user classes (maybe) +* Tested with a micro:bit profile (maybe - very much a personal would like) + +This list excludes the ones that have been done, so this list will hopefully +get shorter, not longer. + + +### Initial 1.x.x version + +The initial version of 1.x.x will NOT be 1.0.0. + +If the last release *before* the 1.x.x release was (for example) 0.5.56, +then the first 1.x.x release would either be 1.5.57 or 1.6.57 + +This is a natural consequence of the fact that MINOR always increases, and +only does as per rules above. It's also a consequence of the fact that the +RELEASE number always increases. + + +### Releases after 1.x.x + +After the first 1.x.x release, the project will use the core rules of +semantic versioning which are but modified slightly: + +*Given a version number MAJOR.MINOR.PATCH, increment the:* + +* *MAJOR version when you make incompatible API changes* +* *MINOR version when you add functionality in a backwards-compatible manner* +* *PATCH version for every release. If just the PATCH version changes, it means bug fixes.* + +(This differs from slightly Semver, but keeps the ordering guarantees, but +it also keeps concepts about history intact) + +So to be clear: + +* If MAJOR changes from 1 to 2, it means there are changes which are not + backwards compatible. This could include changes to the language. Such + versions will obviously be clearly flagged. +* If MINOR changes, it means there's new functionality. In particular, if + there were two versions 1.7.x and 1.9.x, then code developed for 1.7.x + should work with 1.9.x. However code developed for 1.9.x may not work + with 1.7.x + + +### Parallel Releases + +It's possible at some later point in time there may be parallel releases. + +For example there may be a 0.5.56 and a 1.6.57 release. It might be that +testing of an ideas and implementations continues in the 0.X.X release +chain. (Reasoning: 0.x.x is considered unstable by definition) + +This naturally also leads to the concepts of stable releases and unstable +releases. + +This means the release schedule *could* be: + +* 0.5.56 - last dev release before 1.x.x release +* 1.6.57 - First stable 1.x.x release +* 0.6.58 - First dev release after 1.6.57, continuing development, adding new features +* 0.6.59 - Next dev release adding new features +* 0.7.60 - Next dev release adding new features +* 0.7.61 - Next dev release fixing bugs +* 1.7.62 - Second stable 1.x.x release +* 0.7.63 - Next dev release fixing bugs +* 0.7.64 - Next dev release adding features +* 0.8.65 - Next dev release adding major functionality +* 0.9.65 - Next dev release adding more major functionality +* 0.9.66 - Next dev release - candidate stable release perhaps. +* 2.9.67 - This stable release - adds major functionality, but in a + backwards incompatible fashion, so MAJOR version bumps. + +*(Remember, the above is hypothetical!)* + +This actually then maintains the ordering guarantees, and also allows people +to track the relationship of releases to dev releases, and how each release +relates to both previous stable releases *and* development releases. + +Specifically, the stable releases would look like this: + +* 1.6.57 - First stable 1.x.x release +* 1.7.62 - Second stable 1.x.x release +* 2.9.67 - This stable release - adds major functionality, but in a + backwards incompatible fashion, so MAJOR version bumps. + +Note that this maintains the strict ordering requirement we want/need. + +The dev releases would look like this: + +* 0.5.56 - last dev release before 1.x.x release +* 0.6.58 - First dev release after 1.6.57, continuing development, adding new features +* 0.6.59 - Next dev release adding new features +* 0.7.60 - Next dev release adding new features +* 0.7.61 - Next dev release fixing bugs +* 0.7.63 - Next dev release fixing bugs +* 0.7.64 - Next dev release adding features +* 0.8.65 - Next dev release adding major functionality +* 0.9.65 - Next dev release adding more major functionality +* 0.9.66 - Next dev release - candidate stable release perhaps. + +Note again, that this maintains the strict ordering requirement we want/need. + + +### YAGNI? + +This could seem like thinking things through too much, and "You Ain't Gonna +Need It". In a way that's potentially true. + +However, version numbers are essential for signalling to users of a package +the status, and they're also vital for package managers to manage clearly +which version supercedes another. You also do not want a user to +accidentally use an unstable version. + +Also, to be fair, this is the sort of thinking I've had in my head for +versioning based on semver for a while, so writing it down seemed like a +good idea. At some point it might be necessary to change this. + diff --git a/doc/WIPNOTES/6.Models.md b/doc/WIPNOTES/6.Models.md new file mode 100644 index 0000000..4fff3a2 --- /dev/null +++ b/doc/WIPNOTES/6.Models.md @@ -0,0 +1,137 @@ +--- +Summary: Models used by Pyxie +Updated: November 2016 +git-hash: +--- +Pyxie uses a number of models for different purposes + +## iiNodes - Abstract Syntax Tree -- NEEDS IMPLEMENTING PROPERLY + +Designed to be an abstract representation of the program - which is created from pynodes, and designed +to make it simpler to create Cnodes. + +base nodes: + + class iiNode(object): + +Structural + + class iiProgram(iiNode): + +Operators + + def iiOperator(operator): + class iiOpNode(iiNode): + class iiOpPlus(iiOpNode): + class iiOpMinus(iiOpNode): + class iiOpMultiply(iiOpNode): + class iiOpDivide(iiOpNode): + class iiOpBooleanOr(iiOpNode): + class iiOpBooleanAnd(iiOpNode): + class iiOpBooleanNot(iiOpNode): + +Statements + + class iiAssignment(iiNode): + + class iiForStatement(iiNode): + class iiWhileStatement(iiNode): + class iiBreakStatement(iiNode): + class iiContinueStatement(iiNode): + + class iiIfStatement(iiNode): + class iiElifClause(iiNode): + class iiElseClause(iiNode): + + class iiFunctionCall(iiNode): + class iiExpressionStatement(iiNode): + class iiPassStatement(iiNode): + + class iiDefStatement(iiNode): + + class iiComparison(iiNode): + class iiIterator(iiNode): + class iiIdentifierDeclaration(iiNode): + + +Values: + + class iiAttributeAccess(iiNode): + class iiIdentifier(iiNode): + class iiString(iiNode): + class iiInteger(iiNode): + class iiFloat(iiNode): + class iiBoolean(iiNode): + + +## cppnodes - Concrete Syntax Tree -- NEEDS MASSIVE IMPROVEMENT... + +These are designed to represent a C program which can be generated directly from these. + + +## Pynodes - Concrete Syntax Tree + +These are created by the parser front end, and directly represent python programs. +They're divided into the following *organisational* categories: + +pynodes/base_nodes.py: + class PyNode(object): + class PyOperation(PyNode): + class PyStatement(PyNode): + +pynodes/structural.py: + class PyBlock(PyNode): + class PyExprList(PyNode): + class PyProgram(PyNode): + class PyStatements(PyNode): + +pynodes/operators.py: + class PyAndOperator(PyBoolOperator): + class PyBoolOperator(PyOperation): + class PyComparisonOperator(PyOperation): + class PyDivideOperator(PyOperator): + class PyMinusOperator(PyOperator): + class PyNotOperator(PyBoolOperator): + class PyOperator(PyOperation): + class PyOrOperator(PyAndOperator): + class PyPlusOperator(PyOperator): + class PyPowerOperator(PyOperator): + class PyTimesOperator(PyOperator): + +pynodes/statements.py: + class PyAssignment(PyStatement): + + class PyForLoop(PyStatement): + class PyWhileStatement(PyStatement): + class PyBreakStatement(PyStatement): + class PyContinueStatement(PyStatement): + + class PyIfStatement(PyStatement): + class PyElIfClause(PyStatement): + class PyElseClause(PyStatement): + + class PyFunctionCall(PyStatement): + class PyExpressionStatement(PyStatement): + class PyPassStatement(PyStatement): + + class PyEmptyStatement(PyStatement): + class PyDefStatement(PyStatement): + +pynodes/values.py: + class ProfilePyNode(PyIdentifier): + class PyAttributeAccess(PyNode): + class PyAttribute(PyNode): + class PyBinary(PyNumber): + class PyBoolean(PyValueLiteral): + class PyCharacter(PyValueLiteral): + class PyFloat(PyNumber): + class PyHex(PyNumber): + class PyIdentifier(PyValueLiteral): + class PyInteger(PyNumber): + class PyNumber(PyValueLiteral): + class PyOctal(PyNumber): + class PySignedLong(PyNumber): + class PyString(PyValueLiteral): + class PyUnSignedLong(PyNumber): + class PyValueLiteral(PyNode): + diff --git a/doc/changelog.md b/doc/changelog.md index 409d5f0..8e2bffb 100644 --- a/doc/changelog.md +++ b/doc/changelog.md @@ -3,7 +3,97 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/), within reason. (Prior to a 1.0.0 release minor version increases may be backwards -incompatible) +incompatible) See doc/Versioning.md for more details + +## [0.1.26] - UNRELEASED + +### New + +* + +### What's been fixed? + +* + +### Internal Changes + +* + +### Other + +* + +## [0.1.25] - 2018-02-03 + +The focus of this release is the result of the independent intermediate +node refactoring project I was undertaking. This was to transform the list +oriented data structure for representing a "pure" intermediate form of +code into a more stuctured/generalisable format. + +The motivation behind this change was the realisation that implementation +of end user functions (ie `def`) would be very difficult with this more +structured approach. + +### New + +* `doc/Versioning.md` - semantic versioning as it applies to pyxie +* `doc/WIPNOTES/6.Models.md` - start of some docs around the models in use +* Added explicit notes on licensing of pyxie's output. + (Short version: I view the output as being derived from your code by you) +* Language focussed examples / acceptance tests added: + `if` `if-else` `pass` `print` `while-break` `while-continue` +* Change `arduino` profile to support the `Adafruit_NeoPixel` library +* `neopixel` example added + +### What's been fixed? / Improved + +* Handling if if/elif/else improved/fixed +* added `clean.sh` to `arduino` example +* added `clean.sh` to `servo` example +* added `README.md` to `simplest_function` example (won't work yet) + +### Internal Changes + +* `bin/pyxie` now pulls in functionality from `pyxie.api` to be clearer + about what the API is +* added `pyxie/api.py` - public API for embedding pyxie. (Assumes + ubuntu host) Core contents: + * `initialise_API(profile="default")` - Call first + * `set_profile(profile_name)` - Call later + * `PyxieAPI.parse(filename)` - parse file, output goes to console + * `PyxieAPI.analyse(filename)` - parse & analyse flle, output to console + * `PyxieAPI.codegen(filename, result_filename=None)` - parse through + code generation. Output to console + * `PyxieAPI.compile(filename, result_filename=None)` - Compile file, + result ends up as "result_filename" +* `pyxie/core` - changed over to drive the conversion of pynodes to + cppnodes via iinodes. (aim is to simplify pynodes down the line and + make iinodes our main location for analysis/etc) +* Minor changes to `.gitignore` +* Minor change to the `Makefile` to make editting simpler for me... +* Update `clib.py` based on changes to `clib` +* `pyxie/codegen/simple_cpp.py` - functionality shifted out to + `pyxie/models/cppnodes.py` to allow a model driven approach +* `pyxie/model/cppnodes.py` Created. A better code representation + model for C++. Code generation therefore means transforming from the + `iiNode`s to `CppNode`s +* `pyxie/model/iinodes.py` - Introduces `iiNode`s - which are used to + represent an intermediate version of a program such that it can bridge + the conversion between languages +* `pyxie/model/pynodes/operators.py` - added `args()` method to a number + of classes to unify behaviours +* `pyxie/model/transform.py` - Major change - converts `PyNode`s to + `iiNode` representation rather than json objects. + +### Other + +* `doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md` corrected release date +* `doc/newsletter/XX-TBD.Pyxie-2017-Goals.md` - Unreleased newsletter. + Interesting notes, will get reused if appropriate later. Rather moot + contents now though. +* `doc/newsletter/08-TBD.Focus-Or-Pyxie-Release-X.X.XX.md` Template for + next newsletter + ## [0.1.24] - 2016-11-10 @@ -66,7 +156,6 @@ incompatible) * Add special case for function calls like print #37 - ## [0.1.23] - 2016-10-11 Not backwards compatible due to changes to "print" diff --git a/doc/detailed_diff_v0.1.24_v0.1.25.txt b/doc/detailed_diff_v0.1.24_v0.1.25.txt new file mode 100644 index 0000000..3a9f967 --- /dev/null +++ b/doc/detailed_diff_v0.1.24_v0.1.25.txt @@ -0,0 +1,579 @@ +diff --git a/CHANGELOG b/CHANGELOG + + Changelog updated with changes - but as a raw log dump, which isn't + sufficient really (well, it'd do but not really the right approach IMO) + +### New + +* *doc/Versioning.md* Document describing semantic versioning as it applies to pyxie +* doc/WIPNOTES/6.Models.md - start of some documentation around the models in use +* Added explicit notes on licensing of pyxie's output. (Short version: I view the output as being derived from your code by you) +* Language focussed examples / acceptance tests added: if if-else pass print + while-break while-convert_continue_statement +* Change arduino profile to support the Adafruit_NeoPixel library +* Neopixel example added + +### What's been fixed? / Improved + +* Handling if if/elif/else improved/fixed +* added clean.sh to arduino example +* added clean.sh to servo example +* added README.md to simplest_function example (NB, not expected to compile etc yet) + +### Internal Changes + +* bin/pyxie now pulls in functionality from pyxie.api to be clearer about what the API is +* added pyxie/api.py - public API for embedding pyxie. (Assumes ubuntu host) Core contents: + * initialise_API(profile="default") # Call first + * set_profile(profile_name) # Call later + * PyxieAPI.parse(filename) # parse file, output goes to console + * PyxieAPI.analyse(filename) # parse & analyse flle, output to console + * PyxieAPI.codegen(filename, result_filename=None) # parse through code generation. Output to console + * PyxieAPI.compile(filename, result_filename=None) # Compile file, result ends up as "result_filename" +* pyxie/core - changed over to drive the conversion of pynodes to cppnodes via iinodes. (aim is to simplify pynodes down the line and make iinodes our main location for analysis/etc) +* Minor changes to .gitignore +* Minor change to the Makefile to make editting simpler for me... (yeah, I know) +* Update clib.py based on changes to clib +* pyxie/codegen/simple_cpp.py - functionality shifted out to pyxie/models/cppnodes.py to allow a model driven approach +* pyxie/model/cppnodes.py Created. Introduces a better code representation model for C++. + Code generation therefore means transforming from the iiNodes to CppNodes + This change enables this to me model driven. +* pyxie/model/iinodes.py - Introduces iiNodes - which are used to represent an intermediate version of a program such that it can bridge the conversion between languages +* pyxie/model/pynodes/operators.py - added args() method to a number of classes to unify behaviours +* pyxie/model/transform.py - Major change - converts PyNodes to iiNode representation rather than json objects. + +### Other + +* doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md - correct release date +* doc/newsletter/08-TBD.Pyxie-2017-Goals.md - Unreleased newsletter. Interesting notes, will get reused if appropriate later. Rather moot contents now though. +* doc/newsletter/09-TBD.Focus-Or-Pyxie-Release-X.X.XX.md Template for next release + +@@ ############################################################################# + +diff --git a/.gitignore b/.gitignore + + Ignore pyc files and backup files. + Ignore the doc/WIPNOTES/0.RandomDevNotes.md file + +diff --git a/COPYING.output.md b/COPYING.output.md + + Added a file regarding licensing the output of generated code. This is due to seeing lack of an explicit license in the case of Bison causing issues in the past. See the file for more details. + +diff --git a/bin/pyxie b/bin/pyxie + + Modified to pull in functionality from pyxie.api rather than pyxie.core directly + + The aim of this is in part to show how to use pyxie.api, and also to have better high level structure. + + Could still be improved, but better than it was. + +diff --git a/clib/iterators.cpp b/clib/iterators.cpp + + Add explicit license data block for usage with programs' output + +diff --git a/clib/iterators.hpp b/clib/iterators.hpp + + Add explicit license data block for usage with programs' output + +diff --git a/doc/Versioning.md b/doc/Versioning.md + + Add a document detailing how semantic versioning is applied within this project. + In paticular how to differentiate between stable and development versions while + still tracking between the two. + + Istill need a better way of summarising it, but anything beginning with 0 is + development. Anything beginning with 1 (or more) is stable in some shape or form. + Otherwise all version numbers always increase. + +diff --git a/doc/WIPNOTES/6.Models.md b/doc/WIPNOTES/6.Models.md + + Added a WIPNOTE regarding the models that are in use in the system. + This is the start of some internal documentation. It's largely ongoing + notes for development more than anything though. + + Should also help with improving the code structure. + +diff --git a/doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md b/doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md + + Release date corrected in local copy of newsletter + +diff --git a/doc/newsletter/08-TBD.Pyxie-2017-Goals.md b/doc/newsletter/08-TBD.Pyxie-2017-Goals.md + + Un-posted newsletter from the start of 2017 + Covered - successes of 2016 + Listed a number of featues that I consider missing from Pyxie: + * lists + * dictionaries + * tuples + * Actually useful string support + * User functions - ie from def statements + * Generators + * User objects/classes - ie class statements + + Language Goals for 2017 + * JSON level of data support + * User functions + * Classes + + Usage goals + * Ability to run/use perceptrons on an Atmel 8A based robot + + Summarised the releases of 2016 + + Covered the idea of a roadmap for growing pains - specifically the issue of iinodes. + + Probably more useful as blog material now. + +diff --git a/doc/newsletter/09-TBD.Focus-Or-Pyxie-Release-X.X.XX.md b/doc/newsletter/09-TBD.Focus-Or-Pyxie-Release-X.X.XX.md + + Added template for next newsletter. + + This can/should be renamed a slightly different number (8.1 perhaps for filename) + + May be bumped after next release + +diff --git a/examples/arduino/clean.sh b/examples/arduino/clean.sh + + Added clean.sh to arduino example + +diff --git a/examples/servo/clean.sh b/examples/servo/clean.sh + + servo example: Add clean.sh file + +diff --git a/examples/simplest_function/README.md b/examples/simplest_function/README.md + + simplest_function example: Add README.md file + +@@ NEW EXAMPLES + + All of the following have at least a main file, and then scripts for analyse.sh, clean.sh, parse.sh + * examples/if-else + * examples/if + * examples/neopixel + * examples/pass + * examples/print + * examples/while-break + * examples/while-continue + +diff --git a/pyxie/api.py b/pyxie/api.py + + New file - a public API for using pyxie as a code library rather than needing to shell out. + (Could allow things like integration into an editor/IDE/etc) + + API: + + from pyxie.api import initialise_API + from pyxie.api import set_profile + from pyxie.api import PyxieAPI + + initialise_API(profile="default") + # Initialises the API - must be called before the other options + + set_profile(profile_name) + # If after using the API you decide you want to change the profile, you can, + # using this option. + + PyxieAPI.parse(filename) + # parses the given filename, outputs result to console + + PyxieAPI.analyse(filename) + # parses and analyses the given filename, outputs result to console + + PyxieAPI.codegen(filename, result_filename=None) + # parses, analyse and generate code for the given filename, outputs result + # to console. Does not attempt compiling + + PyxieAPI.compile(filename, result_filename=None) + # compiles the given file. + # for a given file "some/program.pyxie" it is compiled to "some/program" + # result_filename can be provide an alternative output name + +diff --git a/pyxie/codegen/clib.py b/pyxie/codegen/clib.py + + Update clib.py based on changes to clib + +diff --git a/pyxie/codegen/simple_cpp.py b/pyxie/codegen/simple_cpp.py + +Added: + def get_blank_line(): + from pyxie.model.cppnodes import CppProgram + +Removed: (Largely shifted into pyxie/models/cppnodes.py) + def mkStatement(statement_spec): + class C_Program(object): + class C_Frame(object): + class Identifier(object): + class Assigment(object): + class PassStatement(object): + class BreakStatement(object): + class ContinueStatement(object): + class ExpressionStatement(object): + def todo(*args): + class ArgumentList(object): + class FunctionCall(object): + class PrintStatement(object): + class WhileStatement(object): + class ForStatement(object): + class IfStatement(object): + class ElIfClause(object): + class ElseClause(object): + def build_program(json): + +diff --git a/pyxie/core.py b/pyxie/core.py + +Added: + Uses iiNode from pyxie.model.iinodes + Added the code header that gets added to generated pyxie code. + def jsonify(node): -- "Takes an iinode based representation and jsonifies it - which is the old (relatively human readable) representation of state + +Changed: + C++ Code generation side now uses a model driven approach, and so pulls in that alternative interface + + generate_code(cname, AST, profile, debug=False): + The key point here is that the intermediate representation between the PyNode AST and the Cpp code generation is iiNodes, rather than a JSON representation. Pretty major change internally. Small external + footprint though. + prepends the new header to the generated C++ code. + +diff --git a/pyxie/model/cppnodes.py b/pyxie/model/cppnodes.py + + Created. Introduces a better code representation model for C++. + Code generation therefore means transforming from the iiNodes to CppNodes + This change enables this to me model driven. + Not ideal, could do with further refactoring + + Interesting things: + * class CppNode(object): + * Special casing of special case python functions (cf print) + + * class CppProgram(CppNode): + * class CppFrame(CppNode): + * class CppIdentifier(CppNode): + * class CppAssignment(CppNode): + * class CppEmptyStatement(CppNode): + * class CppBreakStatement(CppNode): + * class CppContinueStatement(CppNode): + * class CppFunctionCall(CppNode): + * class CppArgumentList(CppNode): + * class CppExpressionStatement(CppNode): + * class CppPrintStatement(CppNode): + * class CppWhileStatement(CppNode): + * class CppForStatement(CppNode): + * class CppIfStatement(CppNode): + * class CppElseIfClause(CppNode): + * class CppElseClause(CppNode): + +diff --git a/pyxie/model/iinodes.py b/pyxie/model/iinodes.py + + This file contains objects used to represent the independent intermediate + format of the program. + + iiNodes represent an intermediate version of a program. The idea is that this represents the program in its "base" form - such that it can be transformed to a concrete language representation. At the moment the only source language is PyNodes, and the only output form is CppNodes, but this could be changed later down the line. As such iiNodes will be expanded in time so that this is wherre type inference and similar takes place. + + Key points: + * class iiNode(object): + * class iiProgram(iiNode): + * class iiAssignment(iiNode): + * class iiOperator(iiNode): + * class iiFunctionCall(iiNode): + * class iiAttributeAccess(iiNode): + * class iiIdentifier(iiNode): + * class iiString(iiNode): + * class iiInteger(iiNode): + * class iiFloat(iiNode): + * class iiBoolean(iiNode): + * class iiComparison(iiNode): + * class iiWhileStatement(iiNode): + * class iiIterator(iiNode): + * class iiForStatement(iiNode): + * class iiDefStatement(iiNode): + * class iiPassStatement(iiNode): + * class iiBreakStatement(iiNode): + * class iiContinueStatement(iiNode): + * class iiExpressionStatement(iiNode): + * class iiIdentifierDeclaration(iiNode): + * class iiElifClause(iiNode): + * class iiElseClause(iiNode): + * class iiIfStatement(iiNode): + + +diff --git a/pyxie/model/pynodes/operators.py b/pyxie/model/pynodes/operators.py + + Add an "args() method to a number of classes - Minor change to allow getting at + arguments in a uniform manner across *all* operators. + +diff --git a/pyxie/model/transform.py b/pyxie/model/transform.py + + Changed: + Bumped copyright date + + Pull in iinodes - transform pynodes to iinodes instead of json. + Means *LOTS* of internal changes + There may be some bits of code which are holdovers from the + transition. If you see any please fix and raise a pull request! + FIXME: Name of file could perhaps be changed to represent transform + +diff --git a/pyxie/profile/arduino.py b/pyxie/profile/arduino.py + * Added Adafruit_NeoPixel to the the list of types in the profile. + * Added a couple of default variables into the runtime context for the profile. + +@@ ################################################ +@@ RAW CHANGE NOTES +@@ ################################################ + +### RAW NOTES TO SUMMARISE + +SHORTLOG + + Create README.md - NB simplest_function is expected to not compile yet + Update release date to match actual release date + Initial pyxie.models.cppnodes + Re-raise exceptions with original context + Move code modelling C++ programs into pyxie/model/cppnodes + Update code to use new names for CppNodes + Extract all list based IInodes to be constructed into a single file + Move the actual entry points for compiling/etc into an API file to serve as a nascent API + Document explaining what a Pyxie version number means. Specifically in the context of semantic versioning + Start of overview/notes regarding internal model nodes + Clarify license over generated code. (ie disclaim effective ownership) + Add neopixel support to arduino profile, with example + Pyxie Versioning + Trace looking for includes + Snapshot changes to cppnodes + Add newsletter not sent earlier in year + Snapshot of template for newsletter 09 (not sent) + Restart iiNode refactor + Better iiNode baseclass (for now) + Refactored explicit json to iiNodes + Bump copyright on file - to indicate when changed last + Return iiNodes, not json + Start of removal of json version + Remove mkPrint / iiPrintStatement since not part of grammar now + Simple acceptance tests for if and if-else + Example for testing break statement fixes + Example to allow quick checking of continue code generation + Add back in an example that uses print + Add clean.sh to servo example + Add example/test that exercises 'pass' + Add clean.sh script to the arduino demo + Make example encompass more types + Some acceptance tests + Dejsonify the intermediate layer + Move jsonify into the UI layer, for display purposes + Include missing import + Remove use of thunk middleman + New parts of changelog for next release + Convenience target added to makefile + Snapshot update of models WIPNOTE + Ignore random devnotes for now + Remove support for list based proto iiNodes + Change contents of 'if' examples readme to match purpose + Change contents of 'if-else' examples readme to match purpose + Ensure operator expressions compile correctly again + + +LONG LOG + +* Ensure operator expressions compile correctly again + Quite a few changes: + + * pyxie/model/pynodes/operators.py - extended to make it simpler to + extract the arguments for the operators. This could be rationalised + further. + + * pyxie/model/iinodes.py - remove the iiOpNode based operator nodes, and + replace with a single simpler iiOperator class. iiComparison modified + to allow the compator to be also called the operator, harmonising + exceptions. + + * pyxie/model/transform.py - Remove usage of list based iiOperators and + use iiOperators instead. + + * pyxie/model/cppnodes.py - Collection of fixes to make these changes work + as expected. + +* Change contents of 'if-else' examples readme to match purpose + +* Change contents of 'if' examples readme to match purpose + +* Remove support for list based proto iiNodes + This is deliberate to flush out bugs and flushes out issues in the + handling of operators + +* Ignore random devnotes for now + +* Snapshot update of models WIPNOTE + +* Convenience target added to makefile + +* New parts of changelog for next release + +* Remove use of thunk middleman + + The middleman is the mk* functions - these allowed the use of jsonify and + iiNodeNames to co-exist. Since the jsonify and lists are no longer used, + these middleman functions can now disappear. + + There is an open question about iiOperator, but that suggests that + iiOperator is in fact badly named and can be simplified somewhat. + Perhaps the approach to take is to have classes of operators - infix + and prefix, and then to treat these as special cases of function calls + instead. + + That would the simplify the implementation. + + That can wait until the ECS based iiNodes though + +* Include missing import + +* Move jsonify into the UI layer, for display purposes + +* Dejsonify the intermediate layer + + The intermediate layer at this point uses both iiNodes and lists. + This commit bundles up the changes necessary to flip the list based + usage to solely use the iiNode based form. This is necessary to + later support better/more interesting/useful things like better + type inference. + +* Some acceptance tests + +* Make example encompass more types + +* Add clean.sh script to the arduino demo + +* Add example/test that exercises 'pass' + +* Add clean.sh to servo example + +* Add back in an example that uses print + +* Example to allow quick checking of continue code generation + +* Example for testing break statement fixes + +* Simple acceptance tests for if and if-else + +* Remove mkPrint / iiPrintStatement since not part of grammar now + +* Start of removal of json version + +* Return iiNodes, not json + +* Bump copyright on file - to indicate when changed last + +* Refactored explicit json to iiNodes + + Large collection of changes (squashed a branch) + Specific additions: + Add iiProgram iiNode + Add iiOpNode, iiOpPlus, iiOpMinus + Add iiOpMultiply + Add iiOpDivide, iiOpBooleanOr, iiOpBooleanAnd, iiOpBooleanNot + Add iiAssignment + Add iiFunctionCall + Add iiAttributeAccess + Add iiIdentifier + Add iiString + Add iiInteger + Add iiFloat + Add iiBoolean + Add iiComparison, iiPrintStatement + Add iiWhileStatement, iiForStatement, iiIterator + Add iiDefStatement, iiPassStatement + Add iiBreakStatement, iiContinueStatement + Added iiExpressionStatement, iiIdentifierDeclaration + Added iiIfStatement, iiElifClause, iiElseClause + Rearrange order of things in the file + +* Better iiNode baseclass (for now) + +* Restart iiNode refactor + +* Snapshot of template for newsletter 09 (not sent) + +* Add newsletter not sent earlier in year + +* Snapshot changes to cppnodes + +* Trace looking for includes + +* Pyxie Versioning + +* Add neopixel support to arduino profile, with example + +* Clarify license over generated code. (ie disclaim effective ownership) + +* Start of overview/notes regarding internal model nodes + +* Document explaining what a Pyxie version number means. Specifically in + the context of semantic versioning + +* Move the actual entry points for compiling/etc into an API file + to serve as a nascent API + + Why? + + Received a query about importing pyxie/core.py into a file and whether + this was the right thing, wrong thing, good thing, bad thing to do. In + particular the query was based on a desire to use pyxie inside another + project. + + Pyxie's main API has always been intended to be the one exposed by + the actual front end bin/pyxie, and of course the specific language + subset it exposes. On the flipside, the internals of the pyxie.core + are still changing as the implementation improves. As a result exposing + pyxie/core.py as an external API (which then has some measure of + commitment to not change) causes problems. As a result, it made sense + to move the implementation of bin/pyxie into pyxie/api.py - so that + the front end can effectively be used programmatically. + + I'm not certain that the names in that file are correct. However, by + creating a hook that can be used as follows: + + from pyxie.api import initialise_API + from pyxie.api import set_profile + from pyxie.api import PyxieAPI + + initialise_API(profile=default) + # Initialises the API - must be called before the other options + + set_profile(profile_name) + # If after using the API you decide you want to change the profile, + # you can, using this option. + + PyxieAPI.parse(filename) + # parses the given filename, outputs result to console + + PyxieAPI.analyse(filename) + # parses and analyses the given filename, outputs result to console + + PyxieAPI.codegen(filename, result_filename=None) + # parses, analyse and generate code for the given filename, + # outputs result to console. Does not attempt compiling + + PyxieAPI.compile(filename, result_filename=None) + # compiles the given file. + # for a given file some/program.pyxie it is compiled to some/program + # result_filename can be provide an alternative output name + + I think is a good starting point. In particular, this only really + exposes 3 names that can't change. The methods parse, analyse, + codegen and compile are all independent of each other. You can + just call compile if you want. These may be augmented by better + versions as time goes on. + +* Extract all list based IInodes to be constructed into a single file + The idea behind this is to allow us to replace the implementation + of IINodes from being nested lists and dictionaries to things + that better represent the code. In particular so we stop losing + information from the analysis phase... + +* Update code to use new names for CppNodes +* Move code modelling C++ programs into pyxie/model/cppnodes +* Re-raise exceptions with original context +* Initial pyxie.models.cppnodes +* Update release date to match actual release date + +* Create README.md - NB simplest_function is expected to not compile yet + (simplest_function example) + As noted in the file, this example is a WIP, and not expected to run yet. diff --git a/doc/dev-status.md b/doc/dev-status.md index c1339e5..c21972f 100644 --- a/doc/dev-status.md +++ b/doc/dev-status.md @@ -10,38 +10,46 @@ commits. _(This is exported from trello)_ * HEADLINE: PRE-ALPHA * DEV STATE: WORKING (USABLE) -* DEV VERSION: 0.1.23 -* RELEASED: 0.0.22 (25 Sep 2016) +* DEV VERSION: 0.1.26 +* RELEASED: 0.1.25 (2 Feb 2018) * LANGUAGE STATE: BARE* -* FOCUS: Practicalities (Arduino Profile real example) - Get the Puppy/Kitten Robot working +* FOCUS: Basic User Functions, Practicalities (Arduino Profile real example) - Get the Puppy/Kitten Robot working * Newsletter created at http://tinyletter.com/sparkslabs ### WIP +* 297\. Detangle the code transforms out from the cppnodes... +* 18\. Code generation of C literals is muddled up a touch with structural representation #internals +* 293\. Extract the transforms from cppnodes into pyxie.transform.ii2cpp ### Paused -* 227\. .get_type() should be delegated, not rely on internal pynode details. -* Flesh out a micro:bit profile. ### Backlog: Arising -* 259\. Would be nice to remove extraneous brackets round generated C++ expressions +* 292\. Debian packaging should include examples +* 294\. Website should include examples. +* 295\. Move analysis to pyxie.transform.analysis ### Release Backlog: Features -* 0\.1.23 = = = = = = = = = = +* 0\.1.26 = = = = = = = = = = +* 286\. Revise iiNodes to use ECS like ideas +* 288\. Use ECS style iiNodes for type inference and variable collation +* 291\. Rename model.transform to transform.pynodetoiinode +* 32\. C Syntax Tree is a Tree #internals ### Release Backlog: Tasks -* 0\.1.23 - - - - - (tasks) +* 0\.1.26 - - - - - (tasks) +* 0\.1.25 - - - - - (tasks) +* 287\. Make repo more visible * 57\. Blog post: Pyxie structure #website -* Release tasks ### Known Bugs / Anti-features @@ -50,86 +58,125 @@ commits. _(This is exported from trello)_ ### Backlog: Proposed Next -* 196\. Batch Compiler -* 59\. Start thoughts on pyxie-web #reflect #website -* 195\. Functionality of bin/pyxie-dev is in core, not a script -* 3\.5 Analysis code looks for an arduino profile file describing c-types appropriately. -* 3\.6 arduino profile file is propogated with something relating to core functions that read values, to avoid forcing types "manually" -* 213\. v0 no funcs Playful Puppy code analyses -* 214\. v0 no funcs Playful Puppy code generates code -* 215\. v0 no funcs Playful Puppy code compiles, and runs on device correctly +* 271\. Transformation from python AST to C-CST exists/works - for simplest possible function +* 268\. Code generation for very basic functions succeeds. (no args, return values or variables) +* 267\. Analysis of functions not using any variables succeeds (no args, no return values) * 221\. Initial spike support for function definitions. (no args, no return values) +* 273\. Code generation for functions has cross overs for that of identifier definition. +* ----------------------------- * 223\. Spike support for functions with basic arguments. * 222\. Spike support for functions which use local variables +* 251\. Profile Context is usable for defining externally defined methods (for typechecking) +* 227\. .get_type() should be delegated, not rely on internal pynode details. +* 15\. Refactor code generation #refactor #internals +* 16\. Find Variables duplicates effort from the analysis phase, while also relying on results from it #internals +* 17\. Transform step is throwing away data. This seems broken and should perhaps pass through decorated pynodes - or at least retain a reference. #internals +* 58\. Blog Post: Pyxie Decisions #website ### Backlog: Features -* 251\. Profile Context is usable for defining externally defined methods/functions (for typechecking) -* 9\. Add the manpage to the distributed files, into the right places #docs -* 8\. Add updating man page to the makefile #docs -* 197\. Web Editor +* 31\. Features focusses on BARE level functionality +* ----------------------------- +* == NEED ====== +* 38\. Generalise special casing of function calls. #practicalities +* 40\. Core expression code generation for strings #pylang +* 41\. Implement container oriented comparison operators #pylang +* 42\. Implement identity oriented comparison operators #pylang +* 48\. Modulo Operator for Integers #pylang +* ----------------------------- +* == USEFUL ====== +* 33\. Can find_variables(AST) in pyxie.model.transform actually just look in the results of the analysis phase? It should be able to. Building the same structure after all #internals +* 36\. Comments are implemented - would be useful for documented tests more appropriately #pylang +* 28\. Better Error messages for users #pylang #practicalties +* 14\. Code Cleanups #refactor #internals * 4\. Compilation profiles are pluggable -* 11\. MBed compatible compilation profile? (Seeedstudio Arch) -* 12\. MSP430 compatible compilation profile? * 13\. Pyxie compile harness supports custom working directories #practicalities -* 14\. Code Cleanups #refactor #internals -* 15\. Refactor code generation #refactor #internals -* 16\. Find Variables duplicates effort from the analysis phase, while also relying on results from it #internals -* 17\. Transform step is throwing away data. This seems broken and should perhaps pass through decorated pynodes - or at least retain a reference. #internals -* 18\. Code generation of C literals is muddled up a touch with structural representation #internals -* 19\. Website should use responsive CSS #website -* 20\. Website side bar could be better implemented. #website * 21\. Truthiness for values that AND/OR/NOT arguments needs resolving properly in C. (deferred) #pylang * 22\. Truthiness of expressions is explicitly checked/valid - for us in if and while statements and boolean expressions #pylang #internals -* 23\. Website could do with some pictures :-) #website +* 47\. Modulo Operator for Strings #pylang +* ----------------------------- +* == WANT ====== +* Flesh out a micro:bit profile. +* 196\. Batch Compiler +* 197\. Web Editor +* 39\. Variables inside while loops are handled correctly #testing +* ----------------------------- +* == WOULDLIKE ====== * 24\. Currently have 2 shift/reduce conflicts. They're auto-resolved correctly, but could be worth seeing if they could be resolved better. #pylang #internals +* 59\. Start thoughts on pyxie-web #reflect #website +* 9\. Add the manpage to the distributed files, into the right places #docs +* 8\. Add updating man page to the makefile #docs +* 11\. MBed compatible compilation profile? (Seeedstudio Arch) +* 12\. MSP430 compatible compilation profile? * 25\. Consider using CppHeaderParser on the C++ side of things - to inform the code generation side of things #internals #pylang -* 26\. Block structure of generated C Code is pretty/human friendly/readable #internals +* 259\. Would be nice to remove extraneous brackets round generated C++ expressions * 27\. Operations and operators could be unified with a bit of tweaking - using "x.operation", not "x.tag/x.comparison" etc #internals #refactor -* 28\. Better Error messages for users #pylang #practicalties * 29\. Duplication exists within code generation for operators. (cf convert_operator etc) #internals #refactor -* 31\. Features focusses on BARE level functionality -* 32\. C Syntax Tree is a Tree #internals -* 33\. Can find_variables(AST) in pyxie.model.transform actually just look in the results of the analysis phase? It should be able to. Building the same structure after all #internals -* 34\. Function call code supports simplified type definitions for externals. #practicalities * 35\. Unify Boolean operator pynode implementations #refactor #internals -* 36\. Comments are implemented - would be useful for documented tests more appropriately #pylang -* 37\. Add special case for function calls like print... #pylang -* 38\. Generalise special casing of function calls. #practicalities -* 39\. Variables inside while loops are handled correctly #testing -* 40\. Core expression code generation for strings #pylang -* 41\. Implement container oriented comparison operators #pylang -* 42\. Implement identity oriented comparison operators #pylang * 43\. Code generation for integer literals retains original representation where possible (Octal, hex, binary, etc) #practicalities * 44\. Function call code supports C style function prototypes for type definitions #tbd * 45\. Function call code supports name C headers for type definitions #tbd * 46\. Pyxie caches results of C style header parsing #tbd -* 47\. Modulo Operator for Strings #pylang -* 48\. Modulo Operator for Integers #pylang * 49\. Modulo Operator for Floats #pylang +* == NOT PRIORITISED ====== ### Backlog: Tasks * 52\. Review whether context should check types of ALL expressions, rather than just first, and whether we can should try to detect type mismatches #reflect -* 53\. Use https://travis-ci.org/ ? #reflect -* 54\. Review pypi packaging for things we should be doing #reflect #practicalities -* 55\. Create pyxie-service for managing batch compilation services #practicalities +* 272\. Review Core Python Types To Do Next ### Backlog: Website, docs, etc +* 20\. Website side bar could be better implemented. #website +* 50\. Link current test programs on the website, maybe #website * 232\. Would be nice to have a prettier website +* 23\. Website could do with some pictures :-) #website * 233\. Some sort of Logo would be nice -* 58\. Blog Post: Pyxie Decisions #website * 60\. Start thoughts on pyxie-gui #reflect * 30\. Should we allow comments on website? #website -* 50\. Link current test programs on the website, maybe #website +* 19\. Website should use responsive CSS #website ### Features Implemented +* 296\. Remove a large chunk of old extraneous debugging output +* 290\. Rename "codegen" as "transform" +* 0\.1.25 = = = = = = = = = = +* 289\. Reimplement Cpp Operators +* 283\. Delete thunk +* 281\. Create classes for remaining iiNodes (if appropriate) +* 280\. Decorate CppNodes to use thunk to continue functioning using *either* iinodes or lists +* 282\. Shift code to use the actual iiNodes, removing use of thunk +* 279\. Create thunk that can take both iiNode lists or iiNodes to allow transition to iiNodes +* 278\. Create base class iiNode +* 285\. Document SemVer for Pyxie +* 284\. Create a pyxie.api import, to allow the use of Pyxie as an API. +* 277\. Move all creation of iiNode lists into functions in pyxie.models.iinodes +* 276\. Shift CppNodes into new location +* 275\. Rename existing CppNodes to make things clearer +* 274\. Create pyxie.models.cppnodes +* 0\.1.24 = = = = = = = = = = +* Prep Release 0.1.24 +* 269\. Pynode support for def statements +* 270\. Add a callable type +* 266\. Grammar parsing for function definition succeeds +* 265\. Lexing for function definition succeeds +* 37\. Add special case for function calls like print... #pylang +* 34\. Function call code supports simplified type definitions for externals. #practicalities +* 26\. Block structure of generated C Code is pretty/human friendly/readable #internals +* 3\.5 Analysis code looks for an arduino profile file describing c-types appropriately. +* 3\.6 Arduino profile now configured in separate "profile" interface file +* 215\. v0 no funcs Playful Puppy code compiles, and runs on device correctly +* 214\. v0 no funcs Playful Puppy code generates code +* 213\. v0 no funcs Playful Puppy code analyses +* 264\. Check all Pynodes add all sub nodes as children correctly +* 263\. Bug: "elif" not analysing/generating code correctly regarding context +* 262\. BUGFIX: "else" not analysing/generating code correctly regarding context +* 195\. Functionality of bin/pyxie-dev is in core, not a script +* 0\.1.23 = = = = = = = = = = * 258\. Improve Arduino profile documentation * 260\. Arduino Profile supports HIGH,LOW,INPUT,OUTPUT,A0-A7 * 238\. Update grammar on website/in docs to match current grammar. @@ -306,6 +353,8 @@ commits. _(This is exported from trello)_ ### Tasks Done +* 0\.1.23 - - - - - (tasks) +* 0\.1.23 Release tasks * 0\.0.22 - - - - - (tasks) * 239\. 0.0.22 Release tasks * 0\.0.22 - - - - - (tasks) @@ -394,5 +443,10 @@ commits. _(This is exported from trello)_ ### Rejected +* 54\. Review pypi packaging for things we should be doing - Seems OK for now +* 55\. Create pyxie-service for managing batch compilation services - to be folded into Make a Batch Compiler +* 53\. Use https://travis-ci.org/ - No time to do at the moment +* 261\. Consider PEP 484 based type hints - maybe later, but part of point of pyxie is to autodetect types +* 298\. Generated C++ code should a list of fragments and joined at the end. diff --git a/doc/index.md b/doc/index.md index 6a710a4..d619a30 100644 --- a/doc/index.md +++ b/doc/index.md @@ -1,10 +1,10 @@ ## Pyxie -- A Little Python to C++ Compiler -**Latest Release:** [0.1.23](changelog.html) (12/Oct/2016) +**Latest Release:** [0.1.25](changelog.html) (3/Feb/2018) ### What job does / will this do? -The aim of this project is to allow a adults and children to write code in a +The aim of this project is to allow adults and children to write code in a familiar high level language that can then be compiled to run on an arbitrary embedded system - that is devices with very low power CPUs and very little memory. (ie devices too small to host a python interpreter/runtime) @@ -446,6 +446,8 @@ inspired heavily by python introspection) That's quite some time off. Release History: +* 0.1.26 - UNRELEASED - TBD +* 0.1.25 - 2018-02-03 - Major changes to internal representation and some new examples * 0.1.24 - 2016-11-10 - Fix assignment in else/elif, update to support Ubuntu 16.04LTS, initial steps on function/def statement support * 0.1.23 - 2016-10-11 - Major arduino profile improvements, print-as-function not statement * 0.0.22 - 2016-09-25 - Enable ability to use a variety of Arduino boards by using an Makefile.in file diff --git a/doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md b/doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md index 67ce6b2..9fb50cf 100644 --- a/doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md +++ b/doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md @@ -2,7 +2,7 @@ # New Pyxie Release 0.1.24 - Improved variable detection in else/elif, start of def statement implementation -## Released: 11 November 2016 +## Released: 10 November 2016 Hi, diff --git a/doc/newsletter/08-TBD.Focus-Or-Pyxie-Release-X.X.XX.md b/doc/newsletter/08-TBD.Focus-Or-Pyxie-Release-X.X.XX.md new file mode 100644 index 0000000..5003359 --- /dev/null +++ b/doc/newsletter/08-TBD.Focus-Or-Pyxie-Release-X.X.XX.md @@ -0,0 +1,121 @@ +# Sparkslabs Update #8 : New Pyxie Release 0.1.25 - RELEASE FOCUS SUMMARY + +# New Pyxie Release 0.1.25 - RELEASE FOCUS SUMMARY + +## Released: DD Month YYYY + +### In progress [ DELETE THIS SECTION ] + +SPECIFIC IN PROGRESS [ DELETE THIS SECTION ] + +... + +TARGET RELEASE DATE: 30/Nov/2016 [ DELETE THIS SECTION ] + +Target Cards: [ DELETE THIS SECTION ] + + * 221. Initial spike support for function definitions. (no args, no return values) (VERY optimistic for this release) + + * 271. Transformation from python AST to C-CST exists/works - for simplest possible function + +Hi, + + +[ PARA1 OVERVIEW OF RELEASE (delete this) ] + +[ PARA2 MAIN USER FEATURE CHANGES (delete this) ] + +[ PARA3 OTHER USER FEATURE CHANGES (delete this) ] + +[ PARA4 INTERNAL CHANGES (delete this) ] + + + +## Major Changes + +[ If many, pick 3 THIS SECTION MUST FIT ON SCREEN. (delete this) ] + +* + +* + +* + +[ Then overview of rest. (delete this) ] + +Also reviewed the backlog to prune what's already been done, or updated. + +### SPECIFIC PROFILE / INTERNAL CHANGES IF USEFUL (delete this otherwise) + + +## Changelog + +[EXTRACTED FROM MAIN CHANGELOG (delete this) ] + +### What's New? + +* [EXTRACTED FROM MAIN CHANGELOG (delete this) ] + + +### What's been fixed? + +* [EXTRACTED FROM MAIN CHANGELOG (delete this) ] + + +### Internal changes + +* [EXTRACTED FROM MAIN CHANGELOG (delete this) ] + +### Other + +* [EXTRACTED FROM MAIN CHANGELOG (delete this) ] + + +## Feedback? + +As usual, feedback is welcome. What would you like to see? Would you like +more detail, less detail? Suggestions for project direction also very +welcome. + + +### Since last release + +I had this feedback from the last release: + + + +## Finally + +As mentioned before, development of pyxie is strictly on my own time with +my own resources, and therefore development is not as fast as I'd like in +an ideal world. + +If you'd like to help, please get in touch. + +I've considered a kickstarter or IndieGoGo project to speed things up. +I'd also consider a patreon account. If there was one, do you think you +would contribute back that way, if you can't contribute code? + +Whether or not you would consider it yourself, how much would you consider +reasonable to pledge? (This isn't a call to do so, but really an open +question - I do see a reasonable amount of traffic to my site from unique +users clearly looking for a python based mechanism for working with +arduinos natively) + + +## Availability + +This release is available on github, pypi and my ubuntu ppa on launchpad. + +* https://github.com/sparkslabs/pyxie +* https://pypi.python.org/pypi/pyxie/ +* https://launchpad.net/~sparkslabs/+archive/ubuntu/packages + +Dev plans: + +* http://www.sparkslabs.com/pyxie/dev-status.html + +As usual, and and all feedback welcome! + + +Michael. (@sparks_rd - https://twitter.com/sparks_rd) diff --git a/doc/newsletter/XX-TBD.Pyxie-2017-Goals.md b/doc/newsletter/XX-TBD.Pyxie-2017-Goals.md new file mode 100644 index 0000000..41c2e5f --- /dev/null +++ b/doc/newsletter/XX-TBD.Pyxie-2017-Goals.md @@ -0,0 +1,239 @@ +# Sparkslabs Update #8 : Pyxie in 2016/2017 + +# Pyxie in 2016/2017 + +This is the newsletter for sparkslabs projects - primarily pyxie. You signed +up some time back and there's an unsubcribe link in this mail. Hopefully this +remains of interest. + +This newsletter reviews progress on Pyxie in 2016, and describes my aspirations for +pyxie in 2017. Feedback has been useful in the past and often resulted in direct +practical improvements. + +## Recap: What is Pyxie? + +It's a (work in progress) python to C++ compiler, primarily targetting small/tiny +embedded systems. + +Last year I started these newsletters describing status, progress and releases +for Pyxie. The aim was to produce one newsletter per month (max) or per release. +There's so far been 7 newsletters, covering 6 releases. There are currently 16 +subscribers, and I see daily traffic to the website. Thank you for your interest! + +## 2016 Success + +Main pyxie headlines of 2016: + + * Profiles became usable, implementable by others, and practical. + + * It becames possible to compile and use an example targetted around controlling + a robot puppy (Dagu Playful puppy) which has 10 servos, and a number of IR + based sensors. + + * This is 364 lines long and a non-trivial example with a variety of basic/core + language constructs. I think this speaks to the status of the language well. + (After all, pyxie aims for utility over completeness) + + * Pyxie's flavour of python became clearly python3 oriented. This was actually + forced as a practical change due to the fact that "Serial.print" is not valid + code in python2, but is in python3. + +The main aim of 2016 was to get to the stage where the playful puppy could be controlled +by a pyxie python program. + + +There's an awful lot still missing from Pyxie as of Jan 2017 though, including: + + * lists + * dictionaries + * tuples + * Actually useful string support + * User functions - ie from def statements + * Generators + * User objects/classes - ie class statements + + +So, what's the technical aspiration this year? + + * Be able to support JSON style data. Perhaps based support on https://github.com/bblanchon/ArduinoJson + * Users to be able to create their own functions, in following gradations: + * Procedures with no arguments + * Procedures with fixed arguments + * Functions with no arguments, 1 return value + * Functions with fixed arguments, 1 return value + * Classes + +If we can do those we can start doing interesting things. So what's the +technical aspiration? + +I'd like for us to be able to run/use perceptrons on an Atmel 8A based robot +which can be used to control said robot. (For those not aware, perceptrons +are an older, simpler model of neural networks. Less powerful, but interesting) + +What sort of perceptron? + +This sort: + + * https://www.codementor.io/mcorr/tutorials/an-introduction-to-python-machine-learning-with-perceptrons-k7pn85vfi + +Probably not that actual code, but that sort of perceptron. + +Language features you'll see used there, not currently implemented in pyxie include: + + * User functions + * List literals, used as arguments to functions + * User classes, user objects, user object method calls + * Floating point math + * Lists as object attributes + * Augmented assignment. (This isn't necessary for functionality) + +Given this matches our list of things we'd like to do quite well, that therefore +makes this a useful example, as well as fun/interesting. (A robot that learns to +follow a line is more interesting than one that was programmed to do so :-) + + +# Headlines of releases in 2016 + +Regarding the break below - I had health related issues prevented me working on Pyxie +from February through to August. Things started improving around then though :-) +I'm still limited by family, day job and other commitments though. + + +*0.0.19* - 2016-01-31 + * Start of support for pyxie to run under python3 + * Practicalities around C++ code for arduino + * Digial I/O & start of servo control + +*0.0.20* - 2016-08-12 + * Mainly internal refacoring. + * Type inference improvement + * WIPNOTES, Arduino examples updated + * C++ output prettified + +*0.0.21* - 2016-09-17 + * Can control servos - meaning can control simple robots + * Lots of internal changes to support + +*0.0.22* - 2016-09-25 + * Practicality improvement to control which arduino board (etc) gets used + via a Makefile.in file. + +*0.1.23* - 2016-10-11 - Major arduino profile improvements, print-as-function not statement + * Ability to capture/assign result of function calls - so reading sensors and acting + on them becomes possible. + * Better docs & examples - including the Puppy example. + * print is now a function ala python3 (allowing Serial.print to compile) + +*0.1.24* - 2016-11-10 - Fix assignment in else/elif, update to support Ubuntu 16.04LTS, initial steps on function/def statement support + * Bug fix for if/else/elif statements + * Primary dev/release platform is now 16.04LTS, with the PPA having backported + releases. + * Profiles bundled into single files, simplifying creation of new profiles. + * Work started on user functions (parsing only at this stage) + +## Specific Roadmap + +https://github.com/sparkslabs/pyxie/projects/1 + +One of the major issues with the codebase at present is fundamentally a "growing pain". + +Essentially, Pyxie has the following functional elements: + + * Parsing and analysing Python: + * Parse the incoming characters using a Lexer to identify basic features + * Recognise those tokens using a grammer. + * Grammar objects that get recognised are represented using PyNodes. + + * C++ Code generation + * Take a representation of a C++ program + * Walk that representation + * Generate concrete statements + * Save that + + * Compilation + + * Transformation from Python representation to C++ representation + +In the early days every representation was a list or list of lists. The C++ +code was represented by a JSON-able structure. This is flexible in early days +and appropriate when you're figuring out what the structures should be and +should contain. + +This is no longer the case. + +So specifically the above is changing to: + + * Parsing to PyNodes + * Transformation from PyNodes to iiNodes + * Transformation from iiNodes to CppNodes + * Code generation from CppNodes + +Where: + * PyNodes represent a concrete python program. + * iiNodes represent the logical program with all type data captured and analysed + * CppNodes represent a concrete C++ program. + +The point being to preserve information that the JSON structure cannot at present use, +but also to prevent the python front end needing to know about the C++ back end nor +vice versa. + +This is pretty standard compiler stuff, but it's now been forced into needing to exist +to simplify things like type inference and local variables/scoping - required for +functions. + +This by definition means refactoring the core code to support this. + +As a result this is in progress. I'd hoped to complete this before christmas, but time +availability was lower than I'd like, so I'm now aiming for "before Easter" - though +hopefully much sooner. + +See also: https://github.com/sparkslabs/pyxie/projects + +## Feedback? + +As usual, feedback is welcome. What would you like to see? Would you like +more detail, less detail? Suggestions for project direction also very +welcome. + + +### Since last release + +I had this feedback from the last release: + + +## Finally + +During many of the previous newsletters I've discussed the *potential* of funding +development for Pyxie (or just defraying developments costs). It's not easy, and it +changes the dynamic quite substantially. The most likely thing I will set up this +year (probably after April) will be a patreon account, and we'll see. This won't +affect ongoing releases nor any of the goals above. + +It will open up options though! + +The aim here of course is to push development of pyxie forward. At present +development is strictly on my own time with my own resources, and therefore +development is not as fast as I'd like in an ideal world. + +(That said, preferred contributions would be in the form of pull requests :) ) + +If you'd like to help, please get in touch. + + +## Availability + +As always, Pyxie is available on github, pypi and my ubuntu ppa on launchpad. + +* https://github.com/sparkslabs/pyxie +* https://pypi.python.org/pypi/pyxie/ +* https://launchpad.net/~sparkslabs/+archive/ubuntu/packages + +Dev plans: + +* http://www.sparkslabs.com/pyxie/dev-status.html + +As usual, and and all feedback welcome! + + +Michael. (@sparks_rd - https://twitter.com/sparks_rd) + diff --git a/doc/project-status.md b/doc/project-status.md index 0768360..80c7685 100644 --- a/doc/project-status.md +++ b/doc/project-status.md @@ -4,10 +4,10 @@ * HEADLINE: PRE-ALPHA * DEV STATE: WORKING (USABLE) -* DEV VERSION: 0.1.23 -* RELEASED: 0.0.22 (25 Sep 2016) +* DEV VERSION: 0.1.26 +* RELEASED: 0.1.25 (2 Feb 2018) * LANGUAGE STATE: BARE* -* FOCUS: Practicalities (Arduino Profile real example) - Get the Puppy/Kitten Robot working +* FOCUS: Basic User Functions, Practicalities (Arduino Profile real example) - Get the Puppy/Kitten Robot working * Newsletter created at http://tinyletter.com/sparkslabs diff --git a/examples/arduino/clean.sh b/examples/arduino/clean.sh new file mode 100755 index 0000000..baff301 --- /dev/null +++ b/examples/arduino/clean.sh @@ -0,0 +1,8 @@ +#!/bin/bash -x + + +rm -rf build-* +rm *.hex +mkdir -p ~/tmp/clean +mv *~ ~/tmp/clean/ + diff --git a/examples/if-else/README.md b/examples/if-else/README.md new file mode 100644 index 0000000..ac07c78 --- /dev/null +++ b/examples/if-else/README.md @@ -0,0 +1,8 @@ +if-else +------- +This directory was created in response to a bug in the code generation of +else statements, and so this directory was therefore created to test a specific +failure case. + +I'm leaving it here to allow testing for regression. It also acts as a simple +example of how pyxie programs can be written, compiled etc. diff --git a/examples/if-else/analyse.sh b/examples/if-else/analyse.sh new file mode 100755 index 0000000..c930b27 --- /dev/null +++ b/examples/if-else/analyse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino analyse examples/if-else/if_else.pyxie diff --git a/examples/if-else/clean.sh b/examples/if-else/clean.sh new file mode 100755 index 0000000..280bcb4 --- /dev/null +++ b/examples/if-else/clean.sh @@ -0,0 +1,7 @@ +#!/bin/bash -x + + +rm -rf build-* +rm *.hex +mkdir -p ~/tmp/clean +mv *~ ~/tmp/clean/ diff --git a/examples/if-else/compile.sh b/examples/if-else/compile.sh new file mode 100755 index 0000000..6010e5e --- /dev/null +++ b/examples/if-else/compile.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino compile examples/if-else/if_else.pyxie diff --git a/examples/if-else/if_else.pyxie b/examples/if-else/if_else.pyxie new file mode 100644 index 0000000..bdf4bba --- /dev/null +++ b/examples/if-else/if_else.pyxie @@ -0,0 +1,6 @@ +pan = 0 + +if 1< 2: + pan = pan - 5 +else: + pan = pan + 5 diff --git a/examples/if-else/parse.sh b/examples/if-else/parse.sh new file mode 100755 index 0000000..8fd148d --- /dev/null +++ b/examples/if-else/parse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. +./bin/pyxie --profile arduino parse examples/if-else/if_else.pyxie + diff --git a/examples/if/README.md b/examples/if/README.md new file mode 100644 index 0000000..8df4760 --- /dev/null +++ b/examples/if/README.md @@ -0,0 +1,9 @@ +if +-- +This directory was created in response to a bug in the code generation of +else statements, and so this directory was therefore created to test the case +of not having an else. + +This directory was created to test/check this. I'm leaving it here to allow +testing for regression. It also acts as a simple example of how pyxie programs +can be written, compiled etc. diff --git a/examples/if/analyse.sh b/examples/if/analyse.sh new file mode 100755 index 0000000..a59217e --- /dev/null +++ b/examples/if/analyse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino analyse examples/if/if.pyxie diff --git a/examples/if/clean.sh b/examples/if/clean.sh new file mode 100755 index 0000000..280bcb4 --- /dev/null +++ b/examples/if/clean.sh @@ -0,0 +1,7 @@ +#!/bin/bash -x + + +rm -rf build-* +rm *.hex +mkdir -p ~/tmp/clean +mv *~ ~/tmp/clean/ diff --git a/examples/if/compile.sh b/examples/if/compile.sh new file mode 100755 index 0000000..5776b57 --- /dev/null +++ b/examples/if/compile.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino compile examples/if/if.pyxie diff --git a/examples/if/if.pyxie b/examples/if/if.pyxie new file mode 100644 index 0000000..9b59a72 --- /dev/null +++ b/examples/if/if.pyxie @@ -0,0 +1,4 @@ +pan = 0 + +if 1< 2: + pan = pan - 5 diff --git a/examples/if/parse.sh b/examples/if/parse.sh new file mode 100755 index 0000000..3d9f0fc --- /dev/null +++ b/examples/if/parse.sh @@ -0,0 +1,5 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. +./bin/pyxie --profile arduino parse examples/if/if.pyxie diff --git a/examples/neopixel/README.md b/examples/neopixel/README.md new file mode 100644 index 0000000..7bbd0e6 --- /dev/null +++ b/examples/neopixel/README.md @@ -0,0 +1,23 @@ +Neopixel example +================ + +Yes, it's near christmas. This is therefore the ideal time to do this, and +write this. This example shows how to use the Adafruit Neopixel library in +Pyxie. + + +Manually: + 1. Clone https://github.com/adafruit/Adafruit_NeoPixel + 2. Copy (or symlink) the directory into /usr/share/arduino/libraries + 3. Done. + +TO BE DONE: + +Less Manually: + 1. Clone https://github.com/sparkslabs/neopixel-packaging + 2. Run make + 3. Install adafruit-neopixel + +Even less Manually - if you're using Ubuntu 16.04LTS + 1. Make sure you're using my ppa + 2. sudo apt-get install adafruit-neopixel \ No newline at end of file diff --git a/examples/neopixel/analyse.sh b/examples/neopixel/analyse.sh new file mode 100755 index 0000000..8b1847a --- /dev/null +++ b/examples/neopixel/analyse.sh @@ -0,0 +1,8 @@ +#!/bin/bash + +FILE=neo-simple.pyxie + +cd ../.. +export PYTHONPATH=. +./bin/pyxie --profile arduino analyse examples/neopixel/$FILE + diff --git a/examples/neopixel/clean.sh b/examples/neopixel/clean.sh new file mode 100755 index 0000000..280bcb4 --- /dev/null +++ b/examples/neopixel/clean.sh @@ -0,0 +1,7 @@ +#!/bin/bash -x + + +rm -rf build-* +rm *.hex +mkdir -p ~/tmp/clean +mv *~ ~/tmp/clean/ diff --git a/examples/neopixel/codegen.sh b/examples/neopixel/codegen.sh new file mode 100755 index 0000000..165534b --- /dev/null +++ b/examples/neopixel/codegen.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie codegen examples/neopixel/neo-simple.pyxie + diff --git a/examples/neopixel/compile.sh b/examples/neopixel/compile.sh new file mode 100755 index 0000000..19e9436 --- /dev/null +++ b/examples/neopixel/compile.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +FILE=neo-simple.pyxie + +cd ../.. +export PYTHONPATH=. +./bin/pyxie --profile arduino compile examples/neopixel/$FILE diff --git a/examples/neopixel/neo-simple.ino b/examples/neopixel/neo-simple.ino new file mode 100644 index 0000000..de3bb49 --- /dev/null +++ b/examples/neopixel/neo-simple.ino @@ -0,0 +1,42 @@ +// NeoPixel Ring simple sketch (c) 2013 Shae Erisson +// released under the GPLv3 license to match the rest of the AdaFruit NeoPixel library + +#include + +// Which pin on the Arduino is connected to the NeoPixels? +// On a Trinket or Gemma we suggest changing this to 1 + +#define PIN 6 + +// How many NeoPixels are attached to the Arduino? + +#define NUMPIXELS 16 + +// When we setup the NeoPixel library, we tell it how many pixels, and which pin to use to send signals. +// Note that for older NeoPixel strips you might need to change the third parameter--see the strandtest +// example for more information on possible values. + +Adafruit_NeoPixel pixels = Adafruit_NeoPixel(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800); + +int delayval = 500; // delay for half a second + +void setup() { + + pixels.begin(); // This initializes the NeoPixel library. +} + +void loop() { + + // For a set of NeoPixels the first NeoPixel is 0, second is 1, all the way up to the count of pixels minus one. + + for(int i=0;i + +pin = 6 +number_of_pixels = 16 +delayval = 500 + +pixels = Adafruit_NeoPixel(number_of_pixels, pin, NEO_GRB + NEO_KHZ800) + +pixels.begin() + +while True: + for i in range(number_of_pixels): + pixels.setPixelColor(i, pixels.Color(0,150,0)) + pixels.show() + delay(delayval) diff --git a/examples/neopixel/parse.sh b/examples/neopixel/parse.sh new file mode 100755 index 0000000..07a997e --- /dev/null +++ b/examples/neopixel/parse.sh @@ -0,0 +1,8 @@ +#!/bin/bash + +FILE=neo-simple.pyxie + +cd ../.. +export PYTHONPATH=. +./bin/pyxie --profile arduino parse examples/neopixel/$FILE + diff --git a/examples/pass/README.md b/examples/pass/README.md new file mode 100644 index 0000000..c5f0f98 --- /dev/null +++ b/examples/pass/README.md @@ -0,0 +1,9 @@ +else-bug +-------- +This directory was created in response to a bug in the code generation of +else statements - specifically in the code generation of statements using +identifiers. The reason for this was due to lack of propogation of context. + +This directory was created to test/check this. I'm leaving it here to allow +testing for regression. It also acts as a simple example of how pyxie programs +can be written, compiled etc. diff --git a/examples/pass/analyse.sh b/examples/pass/analyse.sh new file mode 100755 index 0000000..b217a44 --- /dev/null +++ b/examples/pass/analyse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino analyse examples/pass/pass.pyxie diff --git a/examples/pass/build.log b/examples/pass/build.log new file mode 100644 index 0000000..59506c0 --- /dev/null +++ b/examples/pass/build.log @@ -0,0 +1,257 @@ +CREATING A CONTEXT +Context.store NAME A0 'A0' VALUE ProfilePyNode(0, 'A0') +CONTEXT {'A0': [ProfilePyNode(0, 'A0')]} +Context.store NAME A1 'A1' VALUE ProfilePyNode(0, 'A1') +CONTEXT {'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')]} +Context.store NAME A2 'A2' VALUE ProfilePyNode(0, 'A2') +CONTEXT {'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A2': [ProfilePyNode(0, 'A2')]} +Context.store NAME A3 'A3' VALUE ProfilePyNode(0, 'A3') +CONTEXT {'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')]} +Context.store NAME A4 'A4' VALUE ProfilePyNode(0, 'A4') +CONTEXT {'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A4': [ProfilePyNode(0, 'A4')]} +Context.store NAME A5 'A5' VALUE ProfilePyNode(0, 'A5') +CONTEXT {'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')]} +Context.store NAME A6 'A6' VALUE ProfilePyNode(0, 'A6') +CONTEXT {'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A6': [ProfilePyNode(0, 'A6')]} +Context.store NAME A7 'A7' VALUE ProfilePyNode(0, 'A7') +CONTEXT {'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A7': [ProfilePyNode(0, 'A7')], 'A6': [ProfilePyNode(0, 'A6')]} +Context.store NAME HIGH 'HIGH' VALUE ProfilePyNode(0, 'HIGH') +CONTEXT {'HIGH': [ProfilePyNode(0, 'HIGH')], 'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A7': [ProfilePyNode(0, 'A7')], 'A6': [ProfilePyNode(0, 'A6')]} +Context.store NAME LOW 'LOW' VALUE ProfilePyNode(0, 'LOW') +CONTEXT {'HIGH': [ProfilePyNode(0, 'HIGH')], 'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A7': [ProfilePyNode(0, 'A7')], 'A6': [ProfilePyNode(0, 'A6')], 'LOW': [ProfilePyNode(0, 'LOW')]} +Context.store NAME INPUT 'INPUT' VALUE ProfilePyNode(0, 'INPUT') +CONTEXT {'HIGH': [ProfilePyNode(0, 'HIGH')], 'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A7': [ProfilePyNode(0, 'A7')], 'A6': [ProfilePyNode(0, 'A6')], 'INPUT': [ProfilePyNode(0, 'INPUT')], 'LOW': [ProfilePyNode(0, 'LOW')]} +Context.store NAME OUTPUT 'OUTPUT' VALUE ProfilePyNode(0, 'OUTPUT') +CONTEXT {'HIGH': [ProfilePyNode(0, 'HIGH')], 'OUTPUT': [ProfilePyNode(0, 'OUTPUT')], 'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A7': [ProfilePyNode(0, 'A7')], 'A6': [ProfilePyNode(0, 'A6')], 'INPUT': [ProfilePyNode(0, 'INPUT')], 'LOW': [ProfilePyNode(0, 'LOW')]} +Context.store NAME NEO_GRB 'NEO_GRB' VALUE ProfilePyNode(0, 'OUTPUT') +CONTEXT {'HIGH': [ProfilePyNode(0, 'HIGH')], 'OUTPUT': [ProfilePyNode(0, 'OUTPUT')], 'NEO_GRB': [ProfilePyNode(0, 'OUTPUT')], 'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A7': [ProfilePyNode(0, 'A7')], 'A6': [ProfilePyNode(0, 'A6')], 'INPUT': [ProfilePyNode(0, 'INPUT')], 'LOW': [ProfilePyNode(0, 'LOW')]} +Context.store NAME NEO_KHZ800 'NEO_KHZ800' VALUE ProfilePyNode(0, 'OUTPUT') +CONTEXT {'HIGH': [ProfilePyNode(0, 'HIGH')], 'NEO_KHZ800': [ProfilePyNode(0, 'OUTPUT')], 'OUTPUT': [ProfilePyNode(0, 'OUTPUT')], 'NEO_GRB': [ProfilePyNode(0, 'OUTPUT')], 'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A7': [ProfilePyNode(0, 'A7')], 'A6': [ProfilePyNode(0, 'A6')], 'INPUT': [ProfilePyNode(0, 'INPUT')], 'LOW': [ProfilePyNode(0, 'LOW')]} +USING PROFILE: default +CREATED COMPARISON OPERATOR < PyInteger(3, 1) PyInteger(3, 2) +EMITTING INDENT LexToken(INDENT,'p',4,22) +EMITTING DEDENT LexToken(DEDENT,'#',6,28) +Illegal character '#' LexToken(error,'#\n',6,28) + +-> REACHED BLOCK +PARSES IF + +--> REACHED IF STATEMENT +WARNING, expression used in a location requiring truthiness +This will generally be OK for bools and integers but need a function for anything else +In particular for strings, lists, dictionaries, tuples and so on +That is a separate card in the backlog though + +parse_file: AST includes [] + +parse_file: Raw Parsed AST ---------------------------------- +PyProgram(PyStatements(PyAssignment(PyIdentifier(1, 'pan'),PyInteger(1, 0),'='), + PyIfStatement(PyComparisonOperator(<, PyInteger(3, 1), PyInteger(3, 2)), PyBlock(PyStatements())))) +---------------------------------- + +ANALYSING PROGRAM +ANALYSING STATEMENTS +ANALYSING ASSIGNMENT +**ANALYSE RIGHT** +ANALYSING VALUE LITERAL integer +****ADD R VALUE**** +Context.store NAME pan 'pan' VALUE PyInteger(1, 0) +CONTEXT {'pan': [PyInteger(1, 0)]} +****ANALYSE LEFT**** +PyIdentifier.analyse pan PyIdentifier(1, 'pan') ['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__info__', '__init__', '__json__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'add_child', 'add_children', 'add_context', 'add_rvalue', 'analyse', 'children', 'classname', 'context', 'get_type', 'lineno', 'name', 'ntype', 'tag', 'tags', 'types', 'value'] +attr identifier PyIdentifier(1, 'pan') children [] [] +attr identifier PyIdentifier(1, 'pan') context 140713157466768 {'tag': 'program', 'profile_context': 140713189210768, 'id': 140713157466768, 'parent': None, 'names': {'pan': [PyInteger(1, 0)]}} +attr identifier PyIdentifier(1, 'pan') lineno 1 1 +attr identifier PyIdentifier(1, 'pan') ntype None None +attr identifier PyIdentifier(1, 'pan') tag 'identifier' identifier +attr identifier PyIdentifier(1, 'pan') types [] [] +attr identifier PyIdentifier(1, 'pan') value 'pan' pan +Context.lookup -pc {'tag': 'PROFILE:arduino', 'profile_context': 140713189210768, 'id': 140713189210768, 'parent': None, 'names': {'A1': [ProfilePyNode(0, 'A1')], 'NEO_KHZ800': [ProfilePyNode(0, 'OUTPUT')], 'NEO_GRB': [ProfilePyNode(0, 'OUTPUT')], 'HIGH': [ProfilePyNode(0, 'HIGH')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A7': [ProfilePyNode(0, 'A7')], 'A6': [ProfilePyNode(0, 'A6')], 'OUTPUT': [ProfilePyNode(0, 'OUTPUT')], 'INPUT': [ProfilePyNode(0, 'INPUT')], 'LOW': [ProfilePyNode(0, 'LOW')]}} +Context.TAG program +Context.lookup pan +Context.lookup [('program', {'pan': [PyInteger(1, 0)]}), ('PROFILE:arduino', {'HIGH': [ProfilePyNode(0, 'HIGH')], 'NEO_KHZ800': [ProfilePyNode(0, 'OUTPUT')], 'OUTPUT': [ProfilePyNode(0, 'OUTPUT')], 'NEO_GRB': [ProfilePyNode(0, 'OUTPUT')], 'A1': [ProfilePyNode(0, 'A1')], 'A0': [ProfilePyNode(0, 'A0')], 'A3': [ProfilePyNode(0, 'A3')], 'A2': [ProfilePyNode(0, 'A2')], 'A5': [ProfilePyNode(0, 'A5')], 'A4': [ProfilePyNode(0, 'A4')], 'A7': [ProfilePyNode(0, 'A7')], 'A6': [ProfilePyNode(0, 'A6')], 'INPUT': [ProfilePyNode(0, 'INPUT')], 'LOW': [ProfilePyNode(0, 'LOW')]})] +Type for lvalue: integer +Type for rvalue: integer +Types match: True +ANALYSING IF BLOCK +analyse expression, and analyse block +ANALYSING comparison_operator +ANALYSING VALUE LITERAL integer +ANALYSING VALUE LITERAL integer +ANALYSING BLOCK +ANALYSING STATEMENTS +codegen_phase: _______________________________________________________________ +codegen_phase: GENERATING CODE + +AST.includes = [] +cvariables [] +CONVERT_STATEMENTS PyAssignment(PyIdentifier(1, 'pan'),PyInteger(1, 0),'=') +nodes.PyValueLiteral PyInteger(1, 0) +here +['assignment', 'pan', '=', '0'] +CONVERT_STATEMENTS PyIfStatement(PyComparisonOperator(<, PyInteger(3, 1), PyInteger(3, 2)), PyBlock(PyStatements())) +WORKING THROUGH IF: PyIfStatement(PyComparisonOperator(<, PyInteger(3, 1), PyInteger(3, 2)), PyBlock(PyStatements())) +CONVERT - convert_comparison PyComparisonOperator(<, PyInteger(3, 1), PyInteger(3, 2)) +CONVERTING LITERAL PyInteger(3, 1) +ARG:: integer +here +CONVERTING LITERAL PyInteger(3, 2) +ARG:: integer +here +crepr_arg1 ['integer', 1] +crepr_arg2 ['integer', 2] + +WORKING THROUGH IF: PyBlock(PyStatements()) +CONVERT_STATEMENTS +WORKING THROUGH IF: [] +('MK_IF_STATEMENT', ) +('MK_IF_STATEMENT', []) +('MK_IF_STATEMENT', None) +('II_IF_STATEMENT C', ) +('II_IF_STATEMENT S', []) +('II_IF_STATEMENT EC', None) +[['assignment', 'pan', '=', '0'], ] +INCLUDES:: [] +GETTING INCLUDES ['assignment', 'pan', '=', '0'] +GETTING INCLUDES +generate_code: CONCRETE C SYNTAX TREE: - - - - - - - - - - - - - - - - - - - - +here +('PROGRAM', {'main': {'c_frame': {'identifiers': [], 'statements': [['assignment', 'pan', '=', '0'], ]}}, 'name': 'pass', 'includes': []}) +{'PROGRAM': {'includes': [], + 'main': {'c_frame': {'identifiers': [], + 'statements': [['assignment', + 'pan', + '=', + '0'], + ]}}, + 'name': 'pass'}} +- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +OK, IF_STATEMENT +OK, IF_STATEMENT +OK, IF STATEMENT +OK, EMPTY STATEMENT +BUILDING FOR PROFILE arduino +0 +codegen_phase: _______________________________________________________________ +codegen_phase: Generated Program + +#include "iterators.cpp" + +void setup() +{ + int pan; + + pan = 0; + if ( (1<2) ) { +;; } ; +} + +void loop() +{ +} + +_______________________________________________________________ +C_CODE::: ['//', '// This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie', '//', "// This file is derived from the user's source code, copyright resides", "// with the author of the user's source code.", '//', '// You can edit this header in any way suitable for your project', '//', '', '\n#include "iterators.cpp"\n\nvoid setup()\n{\n int pan;\n \n pan = 0;\n if ( (1<2) ) { \n;; } ;\n}\n\nvoid loop()\n{\n}\n'] +compile_file: COMPILING examples/pass/pass.pyxie +compile_file: IN examples/pass +compile_file: SOURCEFILE pass.pyxie +compile_file: cname pass +compile_file: result_filename examples/pass/pass +buil------------------------- +Arduino.mk Configuration: +- [AUTODETECTED] CURRENT_OS = LINUX +- [AUTODETECTED] ARDUINO_DIR = /usr/share/arduino +- [COMPUTED] ARDMK_DIR = /usr/share/arduino (relative to Common.mk) +- [AUTODETECTED] ARDUINO_VERSION = 105 +- [DEFAULT] ARCHITECTURE = +- [DEFAULT] VENDOR = arduino +- [DEFAULT] ARDUINO_SKETCHBOOK = +- [BUNDLED] AVR_TOOLS_DIR = /usr/share/arduino/hardware/tools/avr (in Arduino distribution) +- [COMPUTED] ARDUINO_LIB_PATH = /usr/share/arduino/libraries (from ARDUINO_DIR) +- [COMPUTED] ARDUINO_VAR_PATH = /usr/share/arduino/hardware/arduino//variants (from ARDUINO_DIR) +- [COMPUTED] BOARDS_TXT = /usr/share/arduino/hardware/arduino//boards.txt (from ARDUINO_DIR) +- [DEFAULT] USER_LIB_PATH = /libraries (in user sketchbook) +- [DEFAULT] PRE_BUILD_HOOK = pre-build-hook.sh +- [USER] BOARD_TAG = leonardo +- [COMPUTED] CORE = arduino (from build.core) +- [COMPUTED] VARIANT = leonardo (from build.variant) +- [COMPUTED] OBJDIR = build-leonardo (from BOARD_TAG) +- [COMPUTED] ARDUINO_CORE_PATH = /usr/share/arduino/hardware/arduino//cores/arduino (from ARDUINO_DIR, BOARD_TAG and boards.txt) +- [ASSUMED] MONITOR_BAUDRATE = 9600 +- [DEFAULT] OPTIMIZATION_LEVEL = s +- [DEFAULT] MCU_FLAG_NAME = mmcu +- [DEFAULT] CFLAGS_STD = +- [DEFAULT] CXXFLAGS_STD = +- [COMPUTED] DEVICE_PATH = /dev/ttyACM0 (from MONITOR_PORT) +- [DEFAULT] FORCE_MONITOR_PORT = +- [AUTODETECTED] Size utility: AVR-aware for enhanced output +- [COMPUTED] BOOTLOADER_PARENT = /usr/share/arduino/hardware/arduino//bootloaders (from ARDUINO_DIR) +- [COMPUTED] ARDMK_VERSION = 1.5 +- [COMPUTED] CC_VERSION = 4.9.2 (avr-gcc) +------------------------- +mkdir -p build-leonardo +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions iterators.cpp -o build-leonardo/iterators.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -x c++ -include Arduino.h -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions pass.ino -o build-leonardo/pass.o +/usr/share/arduino/hardware/tools/avr/bin/avr-gcc -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 /usr/share/arduino/hardware/arduino//cores/arduino/wiring.c -o build-leonardo/core/wiring.o +/usr/share/arduino/hardware/tools/avr/bin/avr-gcc -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 /usr/share/arduino/hardware/arduino//cores/arduino/WInterrupts.c -o build-leonardo/core/WInterrupts.o +/usr/share/arduino/hardware/tools/avr/bin/avr-gcc -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 /usr/share/arduino/hardware/arduino//cores/arduino/wiring_digital.c -o build-leonardo/core/wiring_digital.o +/usr/share/arduino/hardware/tools/avr/bin/avr-gcc -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 /usr/share/arduino/hardware/arduino//cores/arduino/wiring_pulse.c -o build-leonardo/core/wiring_pulse.o +/usr/share/arduino/hardware/tools/avr/bin/avr-gcc -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 /usr/share/arduino/hardware/arduino//cores/arduino/wiring_analog.c -o build-leonardo/core/wiring_analog.o +/usr/share/arduino/hardware/tools/avr/bin/avr-gcc -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 /usr/share/arduino/hardware/arduino//cores/arduino/wiring_shift.c -o build-leonardo/core/wiring_shift.o +/usr/share/arduino/hardware/tools/avr/bin/avr-gcc -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 /usr/share/arduino/hardware/arduino//cores/arduino/avr-libc/realloc.c -o build-leonardo/core/avr-libc/realloc.o +/usr/share/arduino/hardware/tools/avr/bin/avr-gcc -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 /usr/share/arduino/hardware/arduino//cores/arduino/avr-libc/malloc.c -o build-leonardo/core/avr-libc/malloc.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/IPAddress.cpp -o build-leonardo/core/IPAddress.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/new.cpp -o build-leonardo/core/new.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/WString.cpp -o build-leonardo/core/WString.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/Tone.cpp -o build-leonardo/core/Tone.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/WMath.cpp -o build-leonardo/core/WMath.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/Print.cpp -o build-leonardo/core/Print.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/HID.cpp -o build-leonardo/core/HID.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/Stream.cpp -o build-leonardo/core/Stream.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/USBCore.cpp -o build-leonardo/core/USBCore.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/main.cpp -o build-leonardo/core/main.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/HardwareSerial.cpp -o build-leonardo/core/HardwareSerial.o +/usr/share/arduino/hardware/tools/avr/bin/avr-g++ -MMD -c -std=c++11 -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=105 -D__PROG_TYPES_COMPAT__ -I/usr/share/arduino/hardware/arduino//cores/arduino -I/usr/share/arduino/hardware/arduino//variants/leonardo -Wall -ffunction-sections -fdata-sections -Os -DUSB_VID=0x2341 -DUSB_PID=0x8036 -fno-exceptions /usr/share/arduino/hardware/arduino//cores/arduino/CDC.cpp -o build-leonardo/core/CDC.o +/usr/share/arduino/hardware/tools/avr/bin/avr-ar rcs build-leonardo/libcore.a build-leonardo/core/WInterrupts.o build-leonardo/core/wiring.o build-leonardo/core/wiring_pulse.o build-leonardo/core/wiring_analog.o build-leonardo/core/wiring_digital.o build-leonardo/core/wiring_shift.o build-leonardo/core/avr-libc/realloc.o build-leonardo/core/avr-libc/malloc.o build-leonardo/core/Tone.o build-leonardo/core/Print.o build-leonardo/core/new.o build-leonardo/core/Stream.o build-leonardo/core/WString.o build-leonardo/core/USBCore.o build-leonardo/core/main.o build-leonardo/core/IPAddress.o build-leonardo/core/HID.o build-leonardo/core/WMath.o build-leonardo/core/CDC.o build-leonardo/core/HardwareSerial.o +/usr/share/arduino/hardware/tools/avr/bin/avr-gcc -mmcu=atmega32u4 -Wl,--gc-sections -Os -o build-leonardo/build-1511790120.elf build-leonardo/iterators.o build-leonardo/pass.o build-leonardo/libcore.a -lc -lm +/usr/share/arduino/hardware/tools/avr/bin/avr-objcopy -j .eeprom --set-section-flags=.eeprom='alloc,load' \ + --change-section-lma .eeprom=0 -O ihex build-leonardo/build-1511790120.elf build-leonardo/build-1511790120.eep +/usr/share/arduino/hardware/tools/avr/bin/avr-objcopy -O ihex -R .eeprom build-leonardo/build-1511790120.elf build-leonardo/build-1511790120.hex + +/usr/share/arduino/hardware/tools/avr/bin/avr-size --mcu=atmega32u4 -C --format=avr build-leonardo/build-1511790120.elf +AVR Memory Usage +---------------- +Device: atmega32u4 + +Program: 4206 bytes (12.8% Full) +(.text + .data + .bootloader) + +Data: 155 bytes (6.1% Full) +(.data + .bss + .noinit) + + +d_program: _______________________________________________________________ +build_program: BUILDING examples/pass/build-1511790120 + +build_program: Program to build: +['//', + '// This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie', + '//', + "// This file is derived from the user's source code, copyright resides", + "// with the author of the user's source code.", + '//', + '// You can edit this header in any way suitable for your project', + '//', + '', + '\n#include "iterators.cpp"\n\nvoid setup()\n{\n int pan;\n \n pan = 0;\n if ( (1<2) ) { \n;; } ;\n}\n\nvoid loop()\n{\n}\n'] + +build_program: TIDYING +build_program: /usr/bin/indent examples/pass/build-1511790120/pass.ino + +build_program: Done! + +['pass.ino', 'pass.ino~', 'build-leonardo', 'iterators.cpp', 'iterators.hpp', 'Makefile'] +compile_file: BUILD DIR examples/pass/build-1511790120 +compile_file: RESULT FILE examples/pass/build-1511790120/build-leonardo/build-1511790120.hex +compile_file: DEST FILE examples/pass/pass.hex +compile_file: CWD /home/michael/WIP/Development/pyxie diff --git a/examples/pass/clean.sh b/examples/pass/clean.sh new file mode 100755 index 0000000..280bcb4 --- /dev/null +++ b/examples/pass/clean.sh @@ -0,0 +1,7 @@ +#!/bin/bash -x + + +rm -rf build-* +rm *.hex +mkdir -p ~/tmp/clean +mv *~ ~/tmp/clean/ diff --git a/examples/pass/compile.sh b/examples/pass/compile.sh new file mode 100755 index 0000000..51dd6c9 --- /dev/null +++ b/examples/pass/compile.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino compile examples/pass/pass.pyxie diff --git a/examples/pass/parse.sh b/examples/pass/parse.sh new file mode 100755 index 0000000..a1179f0 --- /dev/null +++ b/examples/pass/parse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. +./bin/pyxie --profile arduino parse examples/pass/pass.pyxie + diff --git a/examples/pass/pass.pyxie b/examples/pass/pass.pyxie new file mode 100644 index 0000000..0f8744e --- /dev/null +++ b/examples/pass/pass.pyxie @@ -0,0 +1,4 @@ +pan = 0 + +if 1< 2: + pass diff --git a/examples/print/README.md b/examples/print/README.md new file mode 100644 index 0000000..9342209 --- /dev/null +++ b/examples/print/README.md @@ -0,0 +1,87 @@ +Analog Example +-------------- + +Spelling note: this is spelt analog, rather than analogue because that's the +spelling in the API. I normally prefer UK spellings for things for obvious +reasons, but in this case consistency makes sense. + +The purpose of this example is to provide a practical test for creating and +testing the following API and language elements: + + * Support for profile defined variables - such as A0 + * Support for profile defined objects - such as Serial + * Support for proifle defined functions - such as analogRead, analogWrite + +Furthermore it (unintentionally) caused the need to turn "print" from being a +separate syntactic value to being a function. This is necessary to support +things like Serial.print. This is a little frustrating because I'd hoped to +deal with this later, but it's such an important change, it needs to happen +sooner rather than later. + + +### Current Source + + analogInPin = A0 + analogOutPin = 9 + sensorValue = 0 + outputValue = 0 + Serial.begin(9600) + randomTest = 0 + randomSeed(analogRead(0)) + + while True: + sensorValue = analogRead(analogInPin) + sensorValue = constrain(sensorValue, 10, 150); + outputValue = map(sensorValue, 0, 1023, 0, 255) + randomTest = random(300) + analogWrite(analogOutPin, outputValue) + Serial.print(millis()) + Serial.print(" : ") + Serial.print("sensor:- ") + Serial.print(sensorValue) + Serial.print(" output:- ") + Serial.print(outputValue) + Serial.print(" random:- ") + Serial.print(randomTest) + Serial.println("--------") + delay(2) + +### Generated C++ + + #include "iterators.cpp" + + void setup() { + int analogInPin; + int analogOutPin; + int outputValue; + int randomTest; + int sensorValue; + + analogInPin = A0; + analogOutPin = 9; + sensorValue = 0; + outputValue = 0; + (Serial).begin(9600); + randomTest = 0; + randomSeed(analogRead(0)); + while (true) { + sensorValue = analogRead(analogInPin); + sensorValue = constrain(sensorValue, 10, 150); + outputValue = map(sensorValue, 0, 1023, 0, 255); + randomTest = random(300); + analogWrite(analogOutPin, outputValue); + (Serial).print(millis()); + (Serial).print(" : "); + (Serial).print("sensor:- "); + (Serial).print(sensorValue); + (Serial).print(" output:- "); + (Serial).print(outputValue); + (Serial).print(" random:- "); + (Serial).print(randomTest); + (Serial).println("--------"); + delay(2); + }; + } + + void loop() { + } diff --git a/examples/print/analyse.sh b/examples/print/analyse.sh new file mode 100755 index 0000000..461ad45 --- /dev/null +++ b/examples/print/analyse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino analyse examples/print/print.pyxie diff --git a/examples/print/clean.sh b/examples/print/clean.sh new file mode 100755 index 0000000..280bcb4 --- /dev/null +++ b/examples/print/clean.sh @@ -0,0 +1,7 @@ +#!/bin/bash -x + + +rm -rf build-* +rm *.hex +mkdir -p ~/tmp/clean +mv *~ ~/tmp/clean/ diff --git a/examples/print/compile.sh b/examples/print/compile.sh new file mode 100755 index 0000000..12e9d63 --- /dev/null +++ b/examples/print/compile.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie compile examples/print/print.pyxie + diff --git a/examples/print/parse.sh b/examples/print/parse.sh new file mode 100755 index 0000000..0b9bd10 --- /dev/null +++ b/examples/print/parse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. +./bin/pyxie --profile arduino parse examples/print/print.pyxie + diff --git a/examples/print/print.pyxie b/examples/print/print.pyxie new file mode 100644 index 0000000..44e158e --- /dev/null +++ b/examples/print/print.pyxie @@ -0,0 +1,16 @@ +#include + +print("Hello World") +print("Game Over") +print(1) +print(1.1) +print(True) +print(False) + +print(1+1) +print(1-1) +print(-1) +print(1/1) +print(True or False) +print(True and True) +print(not False) diff --git a/examples/servo/clean.sh b/examples/servo/clean.sh new file mode 100755 index 0000000..baff301 --- /dev/null +++ b/examples/servo/clean.sh @@ -0,0 +1,8 @@ +#!/bin/bash -x + + +rm -rf build-* +rm *.hex +mkdir -p ~/tmp/clean +mv *~ ~/tmp/clean/ + diff --git a/examples/while-break/README.md b/examples/while-break/README.md new file mode 100644 index 0000000..9342209 --- /dev/null +++ b/examples/while-break/README.md @@ -0,0 +1,87 @@ +Analog Example +-------------- + +Spelling note: this is spelt analog, rather than analogue because that's the +spelling in the API. I normally prefer UK spellings for things for obvious +reasons, but in this case consistency makes sense. + +The purpose of this example is to provide a practical test for creating and +testing the following API and language elements: + + * Support for profile defined variables - such as A0 + * Support for profile defined objects - such as Serial + * Support for proifle defined functions - such as analogRead, analogWrite + +Furthermore it (unintentionally) caused the need to turn "print" from being a +separate syntactic value to being a function. This is necessary to support +things like Serial.print. This is a little frustrating because I'd hoped to +deal with this later, but it's such an important change, it needs to happen +sooner rather than later. + + +### Current Source + + analogInPin = A0 + analogOutPin = 9 + sensorValue = 0 + outputValue = 0 + Serial.begin(9600) + randomTest = 0 + randomSeed(analogRead(0)) + + while True: + sensorValue = analogRead(analogInPin) + sensorValue = constrain(sensorValue, 10, 150); + outputValue = map(sensorValue, 0, 1023, 0, 255) + randomTest = random(300) + analogWrite(analogOutPin, outputValue) + Serial.print(millis()) + Serial.print(" : ") + Serial.print("sensor:- ") + Serial.print(sensorValue) + Serial.print(" output:- ") + Serial.print(outputValue) + Serial.print(" random:- ") + Serial.print(randomTest) + Serial.println("--------") + delay(2) + +### Generated C++ + + #include "iterators.cpp" + + void setup() { + int analogInPin; + int analogOutPin; + int outputValue; + int randomTest; + int sensorValue; + + analogInPin = A0; + analogOutPin = 9; + sensorValue = 0; + outputValue = 0; + (Serial).begin(9600); + randomTest = 0; + randomSeed(analogRead(0)); + while (true) { + sensorValue = analogRead(analogInPin); + sensorValue = constrain(sensorValue, 10, 150); + outputValue = map(sensorValue, 0, 1023, 0, 255); + randomTest = random(300); + analogWrite(analogOutPin, outputValue); + (Serial).print(millis()); + (Serial).print(" : "); + (Serial).print("sensor:- "); + (Serial).print(sensorValue); + (Serial).print(" output:- "); + (Serial).print(outputValue); + (Serial).print(" random:- "); + (Serial).print(randomTest); + (Serial).println("--------"); + delay(2); + }; + } + + void loop() { + } diff --git a/examples/while-break/analyse.sh b/examples/while-break/analyse.sh new file mode 100755 index 0000000..a1dd68f --- /dev/null +++ b/examples/while-break/analyse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino analyse examples/while-break/while-break.pyxie diff --git a/examples/while-break/clean.sh b/examples/while-break/clean.sh new file mode 100755 index 0000000..280bcb4 --- /dev/null +++ b/examples/while-break/clean.sh @@ -0,0 +1,7 @@ +#!/bin/bash -x + + +rm -rf build-* +rm *.hex +mkdir -p ~/tmp/clean +mv *~ ~/tmp/clean/ diff --git a/examples/while-break/compile.sh b/examples/while-break/compile.sh new file mode 100755 index 0000000..11392b1 --- /dev/null +++ b/examples/while-break/compile.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino compile examples/while-break/while-break.pyxie + diff --git a/examples/while-break/parse.sh b/examples/while-break/parse.sh new file mode 100755 index 0000000..5c9de35 --- /dev/null +++ b/examples/while-break/parse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. +./bin/pyxie --profile arduino parse examples/while-break/while-break.pyxie + diff --git a/examples/while-break/while-break.pyxie b/examples/while-break/while-break.pyxie new file mode 100644 index 0000000..2dc428a --- /dev/null +++ b/examples/while-break/while-break.pyxie @@ -0,0 +1,11 @@ + +analogInPin = A0 +analogOutPin = 9 +sensorValue = 0 +outputValue = 0 + +while True: + break + Serial.println("--------") + delay(2) + diff --git a/examples/while-continue/README.md b/examples/while-continue/README.md new file mode 100644 index 0000000..9342209 --- /dev/null +++ b/examples/while-continue/README.md @@ -0,0 +1,87 @@ +Analog Example +-------------- + +Spelling note: this is spelt analog, rather than analogue because that's the +spelling in the API. I normally prefer UK spellings for things for obvious +reasons, but in this case consistency makes sense. + +The purpose of this example is to provide a practical test for creating and +testing the following API and language elements: + + * Support for profile defined variables - such as A0 + * Support for profile defined objects - such as Serial + * Support for proifle defined functions - such as analogRead, analogWrite + +Furthermore it (unintentionally) caused the need to turn "print" from being a +separate syntactic value to being a function. This is necessary to support +things like Serial.print. This is a little frustrating because I'd hoped to +deal with this later, but it's such an important change, it needs to happen +sooner rather than later. + + +### Current Source + + analogInPin = A0 + analogOutPin = 9 + sensorValue = 0 + outputValue = 0 + Serial.begin(9600) + randomTest = 0 + randomSeed(analogRead(0)) + + while True: + sensorValue = analogRead(analogInPin) + sensorValue = constrain(sensorValue, 10, 150); + outputValue = map(sensorValue, 0, 1023, 0, 255) + randomTest = random(300) + analogWrite(analogOutPin, outputValue) + Serial.print(millis()) + Serial.print(" : ") + Serial.print("sensor:- ") + Serial.print(sensorValue) + Serial.print(" output:- ") + Serial.print(outputValue) + Serial.print(" random:- ") + Serial.print(randomTest) + Serial.println("--------") + delay(2) + +### Generated C++ + + #include "iterators.cpp" + + void setup() { + int analogInPin; + int analogOutPin; + int outputValue; + int randomTest; + int sensorValue; + + analogInPin = A0; + analogOutPin = 9; + sensorValue = 0; + outputValue = 0; + (Serial).begin(9600); + randomTest = 0; + randomSeed(analogRead(0)); + while (true) { + sensorValue = analogRead(analogInPin); + sensorValue = constrain(sensorValue, 10, 150); + outputValue = map(sensorValue, 0, 1023, 0, 255); + randomTest = random(300); + analogWrite(analogOutPin, outputValue); + (Serial).print(millis()); + (Serial).print(" : "); + (Serial).print("sensor:- "); + (Serial).print(sensorValue); + (Serial).print(" output:- "); + (Serial).print(outputValue); + (Serial).print(" random:- "); + (Serial).print(randomTest); + (Serial).println("--------"); + delay(2); + }; + } + + void loop() { + } diff --git a/examples/while-continue/analyse.sh b/examples/while-continue/analyse.sh new file mode 100755 index 0000000..fe6749c --- /dev/null +++ b/examples/while-continue/analyse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino analyse examples/while-continue/while-continue.pyxie diff --git a/examples/while-continue/clean.sh b/examples/while-continue/clean.sh new file mode 100755 index 0000000..280bcb4 --- /dev/null +++ b/examples/while-continue/clean.sh @@ -0,0 +1,7 @@ +#!/bin/bash -x + + +rm -rf build-* +rm *.hex +mkdir -p ~/tmp/clean +mv *~ ~/tmp/clean/ diff --git a/examples/while-continue/compile.sh b/examples/while-continue/compile.sh new file mode 100755 index 0000000..5740094 --- /dev/null +++ b/examples/while-continue/compile.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. + +./bin/pyxie --profile arduino compile examples/while-continue/while-continue.pyxie + diff --git a/examples/while-continue/parse.sh b/examples/while-continue/parse.sh new file mode 100755 index 0000000..4bf36c7 --- /dev/null +++ b/examples/while-continue/parse.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +cd ../.. +export PYTHONPATH=. +./bin/pyxie --profile arduino parse examples/while-continue/while-continue.pyxie + diff --git a/examples/while-continue/while-continue.pyxie b/examples/while-continue/while-continue.pyxie new file mode 100644 index 0000000..cc75ee3 --- /dev/null +++ b/examples/while-continue/while-continue.pyxie @@ -0,0 +1,11 @@ + +analogInPin = A0 +analogOutPin = 9 +sensorValue = 0 +outputValue = 0 + +while True: + continue + Serial.println("--------") + delay(2) + diff --git a/pyxie/__init__.py b/pyxie/__init__.py index 03214cf..c97bf2d 100644 --- a/pyxie/__init__.py +++ b/pyxie/__init__.py @@ -19,7 +19,7 @@ ### What job does / will this do? -The aim of this project is to allow a adults and children to write code in a +The aim of this project is to allow adults and children to write code in a familiar high level language that can then be compiled to run on an arbitrary embedded system - that is devices with very low power CPUs and very little memory. (ie devices too small to host a python interpreter/runtime) @@ -237,6 +237,8 @@ Release History: +* 0.1.26 - UNRELEASED - TBD +* 0.1.25 - 2018-02-03 - Major changes to internal representation and some new examples * 0.1.24 - 2016-11-10 - Fix assignment in else/elif, update to support Ubuntu 16.04LTS, initial steps on function/def statement support * 0.1.23 - 2016-10-11 - Major arduino profile improvements, print-as-function not statement * 0.0.22 - 2016-09-25 - Enable ability to use a variety of Arduino boards by using an Makefile.in file @@ -398,7 +400,7 @@ legal statements. -Michael Sparks, November 2016 +Michael Sparks, February 2018 """ diff --git a/pyxie/api.py b/pyxie/api.py new file mode 100644 index 0000000..96c8674 --- /dev/null +++ b/pyxie/api.py @@ -0,0 +1,197 @@ +# +# Copyright 2016 Michael Sparks +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +""" +NOTE WELL: This file is an initial public API for using Pyxie in a progammatic context. + +This is a pre-alpha public API, names are likely to change slightly because they +make slightly less sense in this context + +It should be noted that this is based on what has been (until moving here) in bin/pyxie +as an internal API used for the Pyxie front end. It is moved here in case you wish to +call pyxie directly from an application, but not by "calling" the main script. + +I am NOT clear at this instant if this is a good idea or a bad idea. + +Feedback on this would be a good idea. + +API: + + from pyxie.api import initialise_API + from pyxie.api import set_profile + from pyxie.api import PyxieAPI + + initialise_API(profile="default") + # Initialises the API - must be called before the other options + + set_profile(profile_name) + # If after using the API you decide you want to change the profile, you can, + # using this option. + + + PyxieAPI.parse(filename) + # parses the given filename, outputs result to console + + PyxieAPI.analyse(filename) + # parses and analyses the given filename, outputs result to console + + PyxieAPI.codegen(filename, result_filename=None) + # parses, analyse and generate code for the given filename, outputs result + # to console. Does not attempt compiling + + PyxieAPI.compile(filename, result_filename=None) + # compiles the given file. + # for a given file "some/program.pyxie" it is compiled to "some/program" + # result_filename can be provide an alternative output name + +-- November 2016, Michael + +""" +from __future__ import print_function +from __future__ import absolute_import + +__all__ = [ "initialise_API", # You should call this with a profile name + "StandardOptions", # These are the primary methods you will want + "TestsLauncher", # You may want to call these + "set_profile" # You can use this to change the profile name you're using. + ] + +__API_VERSION__ = "0.1" # Only increases when necessary - matches MAJOR/MINOR for project +__API_MAJOR__ = 0 +__API_MINOR__ = 1 + +import pprint + +import pyxie.parsing.context + +from pyxie.core import parse_file +from pyxie.core import analyse_file +from pyxie.core import compile_file +from pyxie.core import codegen_phase +from pyxie.core import compilation_tests +from pyxie.core import compile_testfile +from pyxie.core import parsing_tests +from pyxie.core import parse_testfile +from pyxie.core import testprogs_dir + + +# This is to support compilation profiles. This may or may not turn out to +# be a good approach. (#3/#3.x) +profile = "default" + +def set_profile(profile_name): + global profile + print("USING PROFILE: ", profile) + profile = profile_name + + +# The purpose of this command line dispatcher is to simplify options handling at the +# expense of being a bit pickier about it. + +class CommandLineDispatcher(object): + @classmethod + def handles(klass, command): + return command.replace("-","_") in [ x for x in dir(klass) if not(x.startswith("_"))] + # + @classmethod + def handle(klass, command, *args): + f = getattr(klass, command.replace("-","_")) + f(*args) + + +class TestsLauncher(CommandLineDispatcher): + @staticmethod + def run_tests(): + """Run all tests""" + parsing_tests() + compilation_tests(profile) + + @staticmethod + def parse_tests(): + """Just run parse tests""" + parsing_tests() + + @staticmethod + def compile_tests(): + """Just run compile tests""" + compilation_tests(profile) + + @staticmethod + def parse(filename): + """Parses a given test given a certain filename""" + AST = parse_testfile(testprogs_dir, filename) + pprint.pprint(AST) + pprint.pprint(AST.__json__()) + + @staticmethod + def compile(filename): + """Parses a given test given a certain filename""" + compile_testfile(testprogs_dir, filename, profile) + + +class StandardOptions(CommandLineDispatcher): + @staticmethod + def parse(filename): + """parses the given filename, outputs result to console""" + AST = parse_file(filename) + pprint.pprint(AST) + pprint.pprint(AST.__json__()) + if 0: + import json + print("Also writing a copy to ", filename+".json") + f = open(filename+".json", "w") + f.write(json.dumps(AST.__json__())) + f.close() + + @staticmethod + def analyse(filename): + """parses and analyse the given filename, outputs result to console""" + analyse_file(filename) + + @staticmethod + def codegen(filename, result_filename=None): + """parses, analyse and generate code for the given filename, outputs result to console. Does not attempt compiling""" + codegen_phase(filename, result_filename, profile) + + @staticmethod + def compile(filename, result_filename=None): + """compiles the given file to path/to/filename -- result_filename can be provide an alternative output name""" + compile_file(filename, profile, result_filename) + + +def initialise_API(profile_name): + # Create a default profile - whether or not it is used. + print("CREATING A CONTEXT") + context = pyxie.parsing.context.Context() + context.tag = "PROFILE: "+ profile_name + + if profile_name == "arduino": + from pyxie.profile.arduino import initialise_profile + initialise_profile(context) + + elif profile_name == "default": + # No profile, no profile specific initialisation + pass + + else: + print("UNKNOWN PROFILE") + context.tag = "PROFILE:"+profile_name + + pyxie.parsing.context.profile_context = context + set_profile(profile_name) + + +PyxieAPI = StandardOptions # Alias StandardOptions, but allows us to change the API if necessary diff --git a/pyxie/codegen/clib.py b/pyxie/codegen/clib.py index 1fd079e..76252e3 100644 --- a/pyxie/codegen/clib.py +++ b/pyxie/codegen/clib.py @@ -4,5 +4,5 @@ # Note: This file is autogenerated from clib, using clib/mk_py_clib.py and the C++ files. # Make edits there not here - changes here will be overwritten! # -files = {'iterators.cpp': '#include "iterators.hpp"\n\nstruct range : public Generator {\n int start;\n int end;\n int step;\n\n int index;\n\n range() : start(0), end(0), step(1), index(0) { };\n range(int end) : start(0), end(end), step(1), index(0) { };\n range(int start, int end) : start(start), end(end), step(1), index(start) { };\n range(int start, int end, int stepsize): start(start), end(end), step(stepsize), index(start) { };\n ~range() { };\n\n virtual int next() {\n GENERATOR_START\n\n while ( step>0 ? index < end : index > end) {\n YIELD(index);\n index = index + step;\n }\n\n GENERATOR_END\n }\n};\n', 'iterators.hpp': '#ifndef PYXIE_ITERATORS_HPP\n#define PYXIE_ITERATORS_HPP\n\n/*\n * Python Style Iterators in C++, based in part on experiment work in Kamaelia\n * Redone to remove use of generators, so that this can be used on Arduino\n * \n */\n\n#define GENERATOR_START if (this->__generator_state == -1) { return __default_value; } switch(this->__generator_state) { default:\n#define YIELD(value) this->__generator_state = __LINE__; return ((value) ); case __LINE__:\n#define GENERATOR_END }; this->__generator_state = -1; return __default_value;\n\ntemplate\nstruct Iterator {\n virtual T next()=0;\n virtual bool completed()=0;\n};\n\ntemplate\nclass Generator : public Iterator {\n protected:\n int __generator_state;\n public:\n T __default_value;\n Generator() : __default_value(T()) { };\n ~Generator() { };\n virtual bool completed() { return __generator_state==-1; };\n};\n\n#endif\n', 'iterators_test.cpp': '/*\n * Test of a C++ version of Python style generators.\n * \n */\n\n#include \n#include "iterators.cpp"\n/*\n\nWhile it may be a little unclear, this C++ program is equivalent\nto this python program:\n\nfor count in range(5):\n print count,\n\nprint\n\nIt doesn\'t do this:\n\nfor(int count=0; count<5; count++) {\n std::cout << count;\n}\nstd::cout << "." << std::endl;\n\nBecause doing it the way we do makes it avoid treating for/range as a\nspecial case, meaning we solve the harder problem first. Optimisations\nspecific to certain code structures can come later.\n\n*/\n\n\nint main(int argc, char* argv[]) {\n // Code to be replaced with a range() style iterator\n int count;\n range range_gen = range(5);\n\n while (true) {\n count = range_gen.next();\n if (range_gen.completed())\n break;\n std::cout << count;\n }\n std::cout << "." << std::endl;\n\n return 0;\n}\n'} +files = {'iterators.cpp': '//\n// This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie\n//\n// This file contains parts of code derived from Pyxie itself. It may be used\n// under the following terms:\n//\n// Permission is hereby granted, free of charge, to any person obtaining\n// a copy of this software and associated documentation files (the "Software"),\n// to deal in the Software without restriction, including without limitation\n// the rights to use, copy, modify, merge, publish, distribute, sublicense,\n// and/or sell copies of the Software, and to permit persons to whom the Software\n// is furnished to do so, subject to the following conditions:\n//\n// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n// SOFTWARE.\n//\n// For more information see: http://www.sparkslabs.com/pyxie/COPYING.output.md\n//\n\n#include "iterators.hpp"\n\nstruct range : public Generator {\n int start;\n int end;\n int step;\n\n int index;\n\n range() : start(0), end(0), step(1), index(0) { };\n range(int end) : start(0), end(end), step(1), index(0) { };\n range(int start, int end) : start(start), end(end), step(1), index(start) { };\n range(int start, int end, int stepsize): start(start), end(end), step(stepsize), index(start) { };\n ~range() { };\n\n virtual int next() {\n GENERATOR_START\n\n while ( step>0 ? index < end : index > end) {\n YIELD(index);\n index = index + step;\n }\n\n GENERATOR_END\n }\n};\n', 'iterators.hpp': '//\n// This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie\n//\n// This file contains parts of code derived from Pyxie itself. It may be used\n// under the following terms:\n//\n// Permission is hereby granted, free of charge, to any person obtaining\n// a copy of this software and associated documentation files (the "Software"),\n// to deal in the Software without restriction, including without limitation\n// the rights to use, copy, modify, merge, publish, distribute, sublicense,\n// and/or sell copies of the Software, and to permit persons to whom the Software\n// is furnished to do so, subject to the following conditions:\n//\n// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n// SOFTWARE.\n//\n// For more information see: http://www.sparkslabs.com/pyxie/COPYING.output.md\n//\n\n#ifndef PYXIE_ITERATORS_HPP\n#define PYXIE_ITERATORS_HPP\n\n/*\n * Python Style Iterators in C++, based in part on experiment work in Kamaelia\n * Redone to remove use of generators, so that this can be used on Arduino\n * \n */\n\n#define GENERATOR_START if (this->__generator_state == -1) { return __default_value; } switch(this->__generator_state) { default:\n#define YIELD(value) this->__generator_state = __LINE__; return ((value) ); case __LINE__:\n#define GENERATOR_END }; this->__generator_state = -1; return __default_value;\n\ntemplate\nstruct Iterator {\n virtual T next()=0;\n virtual bool completed()=0;\n};\n\ntemplate\nclass Generator : public Iterator {\n protected:\n int __generator_state;\n public:\n T __default_value;\n Generator() : __default_value(T()) { };\n ~Generator() { };\n virtual bool completed() { return __generator_state==-1; };\n};\n\n#endif\n', 'iterators_test.cpp': '/*\n * Test of a C++ version of Python style generators.\n * \n */\n\n#include \n#include "iterators.cpp"\n/*\n\nWhile it may be a little unclear, this C++ program is equivalent\nto this python program:\n\nfor count in range(5):\n print count,\n\nprint\n\nIt doesn\'t do this:\n\nfor(int count=0; count<5; count++) {\n std::cout << count;\n}\nstd::cout << "." << std::endl;\n\nBecause doing it the way we do makes it avoid treating for/range as a\nspecial case, meaning we solve the harder problem first. Optimisations\nspecific to certain code structures can come later.\n\n*/\n\n\nint main(int argc, char* argv[]) {\n // Code to be replaced with a range() style iterator\n int count;\n range range_gen = range(5);\n\n while (true) {\n count = range_gen.next();\n if (range_gen.completed())\n break;\n std::cout << count;\n }\n std::cout << "." << std::endl;\n\n return 0;\n}\n'} diff --git a/pyxie/codegen/simple_cpp.py b/pyxie/codegen/simple_cpp.py index 6975fc8..c0f95ea 100755 --- a/pyxie/codegen/simple_cpp.py +++ b/pyxie/codegen/simple_cpp.py @@ -18,677 +18,24 @@ from __future__ import print_function from __future__ import absolute_import -import pprint -import pyxie.model.functions -from pyxie.model.functions import builtins -from pyxie.codegen.profiles import cpp_templates - +from pyxie.model.cppnodes import CppProgram blank_line = "" -unique_id = 0 source = [] + def Print(*args): y = " ".join([str(x) for x in args]) source.append(y) -def reset_parser(): - global source - source = [] - -def mkStatement(statement_spec): - ss = statement_spec - if ss[0] == "assignment": - return Assigment( ss[1], ss[3], ss[2]) - - elif ss[0] == "print_statement": - return PrintStatement(*statement_spec[1:]) - - elif ss[0] == "expression_statement": - return ExpressionStatement(*statement_spec[1:]) - - elif ss[0] == "while_statement": - return WhileStatement(*statement_spec[1:]) - - elif ss[0] == "for_statement": - return ForStatement(*statement_spec[1:]) - - elif ss[0] == "if_statement": - if len(ss) == 3: - return IfStatement(ss[1], ss[2]) - if len(ss) == 4: - return IfStatement(ss[1], ss[2], ss[3]) - raise NotImplementedError("Have not yet implemented else clauses...") - - elif ss[0] == "pass_statement": - return PassStatement() - - elif ss[0] == "break_statement": - return BreakStatement() - - elif ss[0] == "continue_statement": - return ContinueStatement() - - else: - print("Unknown statement type", ss[0], ss) - - -class C_Program(object): - def __init__(self): - self.includes = [] - self.main_cframe = C_Frame() - self.name = "" - - @classmethod - def fromjson(klass, json): - program = klass() - prog_desc = json["PROGRAM"] - program.name = prog_desc["name"] - program.includes = list(prog_desc["includes"]) - main_spec = prog_desc["main"]["c_frame"] - for identifier in main_spec["identifiers"]: - program.main_cframe.identifiers.append(Identifier(identifier[1], identifier[2])) - - for statement in main_spec["statements"]: - conc_statement = mkStatement(statement) - program.main_cframe.statements.append(conc_statement) - - return program - - def generate(self, profile = "default"): - print("BUILDING FOR PROFILE", profile) - frame_lines = self.main_cframe.concrete() - seen = {} - for include in self.includes: - if not seen.get(include, False): - # Only output each include once - Print( "#include "+ include ) - seen[include] = True - - print_def = cpp_templates.get(profile, cpp_templates.get("default")) - frame_text = "\n".join([" "+line for line in frame_lines]) - - Print(print_def % { "FRAME_TEXT": frame_text } ) - - def json(self): - return { "PROGRAM": {"name": self.name, - "includes" : self.includes, - "main": self.main_cframe.json()} - } - -class C_Frame(object): - def __init__(self): - self.identifiers = [] - self.statements = [] - - def json(self): - for y in self.statements: - Print(y) - return {"c_frame": {"identifiers" : [ x.json() for x in self.identifiers ], - "statements" : [y.json() for y in self.statements ] } - } - - def concrete(self): - block = [] - for identifier in self.identifiers: - decl_code = identifier.decl_code() - block.append(decl_code + ";") - - block.append(blank_line) - for statement in self.statements: - if statement: - code = statement.code() - if code is None: - print("STATEMENT IS None, WHY?", statement) - block.append(code + ";") - return block - -class Identifier(object): - def __init__(self, ctype, name): - self.ctype = ctype - self.name = name - - def json(self): - return ["identifier", self.ctype, self.name ] - - def decl_code(self): - return self.ctype + " " + self.name - -class Assigment(object): - def __init__(self, lvalue, rvalue, assigntype="="): - self.lvalue = lvalue - self.rvalue = rvalue - self.assigntype = assigntype - - def json(self): - return ["assignment", self.lvalue, self.assigntype, self.rvalue ] - - def code(self): - print(self.rvalue) - if type(self.rvalue) == list: - print("Aha!",self.rvalue) - crvalue = ArgumentList(self.rvalue) - crvalue = crvalue.code() - print("AHA!", crvalue) - else: - crvalue = self.rvalue - return self.lvalue + " "+self.assigntype+" " + crvalue - -# A pass statement is not a commented out statement. -# -# By that this: -# while True: pass -# -# Is equivalent to while(true); -# As opposed to while(true); -# -# So that's why this returns ";" not "" -class PassStatement(object): - def json(self): - return ["pass_statement"] - - def code(self): - return ";" - - - -class BreakStatement(object): - def json(self): - return ["break_statement"] - - def code(self): - return "break" - -class ContinueStatement(object): - def json(self): - return ["continue_statement"] - - def code(self): - return "continue" - -class ExpressionStatement(object): - def __init__(self, expression): - self.expression = expression - - def json(self): - return ["expression_statement", self.expression ] - - def code(self): - print(self.expression) - if type(self.expression) == list: - cvalue = ArgumentList(self.expression) - cvalue = cvalue.code() - else: - raise Exception("Fail to understand how to generate code for %s" % repr(self.expression) ) - - return cvalue - -def todo(*args): - print("TODO", " ".join([repr(x) for x in args])) - -class ArgumentList(object): - def __init__(self, *args): - self.args = args - - def json(self): - return list(self.args[:]) - - def code_op_F(self, arg): - if arg[1] == "plus": return "+" - if arg[1] == "minus": return "-" - if arg[1] == "times": return "*" - if arg[1] == "divide": return "/" - if arg[1] == "boolean_or": return " || " - if arg[1] == "boolean_and": return " && " - if arg[1] == "boolean_not": return " ! " - - if arg[1] in ["<", ">", "==", ">=", "<=", "!="]: - return arg[1] - - if arg[1] == "<>": return "!=" - - return None - - def code_op(self,arg): - c_op = self.code_op_F(arg) - if c_op: - - if len(arg) == 4: - arg1 = arg[2] - arg2 = arg[3] - # We would like to try to assert here that the values on both sides - # are integers, but at present we cannot do that, since we use too simplistic - # a structure. If I remove that constraint, we can generate more code. - # But we will need to revisit this. - lit_arg1 = self.code_arg(arg1) - lit_arg2 = self.code_arg(arg2) - - result = "(" + lit_arg1 + c_op + lit_arg2 + ")" - return result - if len(arg) == 3: - arg1 = arg[2] - # We would like to try to assert here that the values on both sides - # are integers, but at present we cannot do that, since we use too simplistic - # a structure. If I remove that constraint, we can generate more code. - # But we will need to revisit this. - lit_arg1 = self.code_arg(arg1) - - result = "(" + c_op + lit_arg1 + ")" - return result - - todo("Handle code ops for anything other than plus/int,int") - raise NotImplementedError("Handle code ops for anything other than plus/int,int" + repr(arg)) - - - def code_arg(self, arg): - if arg[0] == "identifier": - return arg[1] - elif arg[0] == "integer": - r = repr(arg[1]) - if arg[1]<0: - r = "(" + r + ")" - return r - elif arg[0] == "string": - carg = arg[1].replace('"', '\\"') - return '"' + carg + '"' # Force double quotes - elif arg[0] == "double": - return repr(arg[1]) - elif arg[0] == "boolean": - return arg[1] - elif arg[0] == "op": - return self.code_op(arg) - elif arg[0] == "function_call": - code_gen = FunctionCall(arg[1],arg[2]) - return code_gen.code() - print("We don't know how to generate code for function calls yet", arg) - return "" - todo("Handle print value types that are more than the basic types") - raise NotImplementedError("Handle print value types that are more than the basic types" + repr(arg)) - - def code_list(self): - cargs = [] - for arg in self.args: - c_str = self.code_arg(arg) - cargs.append(c_str) - - return cargs - - def code(self): - return ",".join(self.code_list()) - -class FunctionCall(object): - def __init__(self, identifier, args): - self.args = args - self.identifier = identifier - self.arg_list = ArgumentList(*args) - - def json(self): - return ["function_call", self.identifier ] + self.arg_list.json() - - def code(self): - identifier = self.identifier[1] - - if self.identifier[0] == "attributeaccess": - expression = self.identifier[1] - attribute = self.identifier[2] - - c_expression = "(" + expression[1] + ")" - c_attribute = attribute[1] - - identifier = c_expression + "." + c_attribute - - else: - identifier = self.identifier[1] - - # FUDGE: The following would need to be profile specific - # - # It probably also needs reviewing/revising later on - if identifier == "print": - X = PrintStatement(*self.args) - return X.code() - arglist = ", ".join(self.arg_list.code_list()) - return identifier + "(" + arglist + ")" +def get_blank_line(): + return blank_line -class PrintStatement(object): - def __init__(self, *args): - self.args = args - self.arg_list = ArgumentList(*args) - - def json(self): - return ["print_statement" ] + self.arg_list.json() - - def code(self): - return "( cout << " + " << \" \" << ".join(self.arg_list.code_list()) + " << endl )" - - -class WhileStatement(object): - def __init__(self, condition, *statements): - self.raw_condition = condition - self.raw_statements = list(statements) - self.block_cframe = C_Frame() - - for statement in self.raw_statements: - conc_statement = mkStatement(statement) - self.block_cframe.statements.append(conc_statement) - - def json(self): - return ["while_statement", self.raw_condition] + self.raw_statements - - def code(self): - code = "while" - code += "(" - code += ArgumentList(self.raw_condition).code() - code += ") {" - code += "\n".join( self.block_cframe.concrete() ) - code += "}" - return code - -class ForStatement(object): - def __init__(self, identifier, iterable, statements, for_statement_pynode): - self.raw_identifier = identifier - self.raw_iterable = iterable - self.raw_statements = list(statements) - self.block_cframe = C_Frame() - self.for_statement_pynode = for_statement_pynode - - for statement in self.raw_statements: - conc_statement = mkStatement(statement) - self.block_cframe.statements.append(conc_statement) - - def json(self): - return ["for_statement", self.raw_condition, self.raw_iterable, self.raw_statements ] - - def code(self): - # - # The following is the sort of code we want to generate inline. - # This actually implements what we're after. - # However it requires some declarations to be in place really of the the find variables kind - # That said, C++ allows inline delcarations like this (for good or ill), so maybe just do - # this in the first instance, and leave improvements for refactoring??? - # - # Actually this discussion is useful, but it turns out to allow nested generator usage - # ie for x in range(5): for y in range(x): print x,y - # It requires this to be done closer to what could be viewed as "properly" - # -# %(ITERATOR_TYPE)s %(ITERATOR_NAME)s = %(ITERATOR_EXPRESSION)s; - template = """ - %(ITERATOR_NAME)s = %(ITERATOR_EXPRESSION)s; - while (true) { - %(IDENTIFIER)s = %(ITERATOR_NAME)s.next(); - if (%(ITERATOR_NAME)s.completed()) - break; - - %(BODY)s // Itself uses %(IDENTIFIER)s - } - """ - global unique_id - unique_id = unique_id+1 - - iterator_ctype = "range" - iterator_expression = self.raw_iterable ## NOTE THIS RESULTS IN ['iterate_over', ['function_call', ['identifier', 'range'], [['integer', 5]]]]- so that needs work - - print("OK, USE THIS THEN", self.for_statement_pynode.expression.ivalue_name) - iterator_name = self.for_statement_pynode.expression.ivalue_name - - if self.raw_iterable[0] == "iterator": - iterable = self.raw_iterable[1] - if iterable[0] == "function_call": - assert iterable[1][0] == "identifier" - identifier = iterable[1][1] - if identifier in builtins: - print("YAY") - pprint.pprint( builtins[identifier] ) - iterator_ctype = builtins[identifier]["iterator_ctype"] - print("iterator_name -- ", iterator_ctype) - print("YAY") - - print(iterable) - fcall = FunctionCall(iterable[1],iterable[2]) - fcall_code = fcall.code() - iterator_expression = fcall_code - - template_args = { "ITERATOR_TYPE": iterator_ctype, - "ITERATOR_EXPRESSION": iterator_expression, - "ITERATOR_NAME": iterator_name, - "UNIQIFIER":repr(unique_id), - "IDENTIFIER": self.raw_identifier, - "BODY": "\n".join( self.block_cframe.concrete() ) - } - - - - print("=== template args =================================================") - pprint.pprint (template_args ) - print("--- template args -------------------------------------------------") - if self.for_statement_pynode.expression.isiterator: - identifier = self.for_statement_pynode.identifier - print(identifier.get_type()) - print("=== end extractable template args =================================") - print(template % template_args) - - print("----------------------------------------------------------") - print(dir(self.for_statement_pynode)) - print(repr(self.for_statement_pynode)) - print("----------------------------------------------------------") -# raise NotImplementedError("Haven't finished implementing ForStatement yet...") - code = template % template_args - - print("CODE", code) - - return code - -class IfStatement(object): - def __init__(self, condition, statements, extended_clause=None): - self.raw_condition = condition - self.raw_statements = list(statements) - self.extended_clause = extended_clause - - self.block_cframe = C_Frame() - for statement in self.raw_statements: - conc_statement = mkStatement(statement) - self.block_cframe.statements.append(conc_statement) - - def json(self): - result = ["if_statement", self.raw_condition, self.raw_statements ] - if self.extended_clause is not None: - result.append(extended_clauses.json()) - - def code(self): - extended_clauses_code = None - condition_code = ArgumentList(self.raw_condition).code() - block_code = "\n".join( self.block_cframe.concrete() ) - if self.extended_clause: - if self.extended_clause[0] == "elif_clause": - print(self.extended_clause[0]) - condition = self.extended_clause[1] - statements = self.extended_clause[2] - if len(self.extended_clause) == 4: - extended_sub_clause = self.extended_clause[3] - extended_clauses_code = ElIfClause( condition, statements, extended_sub_clause ).code() - else: - extended_clauses_code = ElIfClause( condition, statements ).code() - - if self.extended_clause[0] == "else_clause": - print("***************************************************") - print(self.extended_clause[0]) - statements = self.extended_clause[1] - extended_clauses_code = ElseClause( statements ).code() - - code = "if ( %s ) { %s } " % (condition_code, block_code ) - if extended_clauses_code: - code = code + extended_clauses_code - return code - -class ElIfClause(object): - def __init__(self, condition, statements, extended_clause=None): - self.raw_condition = condition - self.raw_statements = list(statements) - self.extended_clause = extended_clause - - self.block_cframe = C_Frame() - for statement in self.raw_statements: - conc_statement = mkStatement(statement) - self.block_cframe.statements.append(conc_statement) - - def json(self): - result = ["elif_clause", self.raw_condition, self.raw_statements ] - if self.extended_clause is not None: - result.append(extended_clause.json()) - - def code(self): - extended_clauses_code = None - condition_code = ArgumentList(self.raw_condition).code() - block_code = "\n".join( self.block_cframe.concrete() ) - if self.extended_clause: - if self.extended_clause[0] == "elif_clause": - print("***************************************************") - print(self.extended_clause[0]) - condition = self.extended_clause[1] - statements = self.extended_clause[2] - - if len(self.extended_clause) == 4: - extended_sub_clause = self.extended_clause[3] - extended_clauses_code = ElIfClause( condition, statements, extended_sub_clause ).code() - else: - extended_clauses_code = ElIfClause( condition, statements ).code() - - - if self.extended_clause[0] == "else_clause": - print("***************************************************") - print(self.extended_clause[0]) - statements = self.extended_clause[1] - extended_clauses_code = ElseClause( statements ).code() - - code = "else if ( %s ) { %s } " % (condition_code, block_code ) - if extended_clauses_code: - code = code + extended_clauses_code - return code - - -class ElseClause(object): - def __init__(self, statements): - self.raw_statements = list(statements) - - self.block_cframe = C_Frame() - for statement in self.raw_statements: - conc_statement = mkStatement(statement) - self.block_cframe.statements.append(conc_statement) - - def json(self): - result = ["elif_clause", self.raw_statements ] - - def code(self): - extended_clauses_code = None - block_code = "\n".join( self.block_cframe.concrete() ) - - code = "else { %s } " % (block_code, ) - - return code - -def build_program(json): - json = C_Program.fromjson(json) - - -if __name__ == "__main__": - - - - if 1: - progj = {'PROGRAM': {'includes': [''], - 'main': {'c_frame': {'identifiers': [], - 'statements': [['print_statement', - ['op', - 'plus', - ['integer', 1], - ['integer', 1]]]]}}, - 'name': 'hello_operators'}} - - program = C_Program.fromjson(progj) - print(program) - - program.generate() - import time - import pprint - import os - - pprint.pprint(source, width=200) - now = int(time.time()) - dirname = str(now - 1427000000) - # Print("BUILDING PROGRAM", dirname) - os.mkdir(dirname) - f = open(os.path.join(dirname,program.name+".c"), "w") - for line in source: - f.write(line) - f.write("\n") - f.close() - - from profiles import makefile_templates - makefile_tmpl = makefile_templates.get("default") - makefile = makefile_tmpl % {"filename": program.name } - f = open(os.path.join(dirname,"Makefile"), "w") - f.write(makefile) - f.close() - - os.chdir(dirname) - os.system("make") - - - - # Build an example concrete syntax tree - if 0: - program = C_Program() - - program.name = ("hello") - program.includes.append("") - program.includes.append("") - - main = program.main_cframe - main.identifiers.append(Identifier("string", "greeting")) - main.identifiers.append(Identifier("string", "name")) - main.statements.append(Assigment("greeting", '"hello"')) - main.statements.append(Assigment("name", '"world"')) - main.statements.append(PrintStatement("greeting", "name")) - - import pprint - - progj = program.json() - - pprint.pprint(progj) - - program_clone = C_Program.fromjson(progj) - progj2 = program_clone.json() - - program = program_clone - - Print("--------------------------------------------------------------") - pprint.pprint(progj2) - - program.generate() - - import time - import pprint - import os - - pprint.pprint(source, width=200) - now = int(time.time()) - dirname = str(now - 1427000000) - Print("BUILDING PROGRAM", dirname) - os.mkdir(dirname) - f = open(os.path.join(dirname,program.name+".c"), "w") - for line in source: - f.write(line) - f.write("\n") - f.close() - - from profiles import makefile_templates - makefile_tmpl = makefile_templates.get("default") - - makefile = makefile_tmpl % {"filename": program.name } - f = open(os.path.join(dirname,"Makefile"), "w") - f.write(makefile) - f.close() +def reset_parser(): + global source + source = [] - os.chdir(dirname) - os.system("make") +# def build_program(json): +# json = CppProgram.fromjson(json) diff --git a/pyxie/core.py b/pyxie/core.py index ea5ef63..e56998a 100644 --- a/pyxie/core.py +++ b/pyxie/core.py @@ -32,11 +32,23 @@ from pyxie.model.transform import ast_to_cst from pyxie.codegen.clib import files as clib_files -from pyxie.codegen.simple_cpp import C_Program, source, reset_parser +from pyxie.codegen.simple_cpp import CppProgram, source, reset_parser + +from pyxie.model.iinodes import iiNode testdir = "test-data" testprogs_dir = os.path.join(testdir, "progs") +code_header = ["//", + "// This file contains code generated by Pyxie - http://www.sparkslabs.com/pyxie", + "//", + "// This file is derived from the user's source code, copyright resides", + "// with the author of the user's source code.", + "//", + "// You can edit this header in any way suitable for your project", + "//", + ""] + def copy_file(source, dest): f = open(source, "rb") @@ -135,14 +147,23 @@ def analyse_file(filename): pprint.pprint(AST.__info__()) print("------------") +def jsonify(node): + if isinstance(node, iiNode): + print ("here") + return node.__json__() + elif ( isinstance(node, list) or isinstance(node, dict) or isinstance(node, str) or isinstance(node, int) or isinstance(node, float) or isinstance(node, bool) ): + return node + return ValueError("Don't know what to do with this value"+repr(node)) + + def generate_code(cname, AST, profile, debug=False): - CST = ast_to_cst(cname, AST) + iiNodes = ast_to_cst(cname, AST) if debug: print("generate_code: CONCRETE C SYNTAX TREE: - - - - - - - - - - - - - - - - - - - -") - #print(pprint.pformat(CST)) - print(json.dumps(CST, indent=4)) + print(pprint.pformat(jsonify(iiNodes))) print("- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -") - program = C_Program.fromjson(CST) + + program = CppProgram.fromiinodes(iiNodes) program.generate(profile) return pyxie.codegen.simple_cpp.source[:] @@ -267,6 +288,9 @@ def compile_file(filename, profile, result_filename=None): c_code = codegen_phase(filename, result_filename, profile) + c_code = code_header + c_code + + print("C_CODE:::", repr(c_code)) print("compile_file: COMPILING", filename) print("compile_file: IN", base_dir) print("compile_file: SOURCEFILE", base_filename) diff --git a/pyxie/devcore.py b/pyxie/devcore.py index eaae86b..ed23f3d 100644 --- a/pyxie/devcore.py +++ b/pyxie/devcore.py @@ -352,7 +352,8 @@ def upload_to_pypi(): # Build PyPI release os.system("rm -rf dist") os.system("rm MANIFEST") - os.system("python setup.py sdist upload") + os.system("python setup.py sdist") + os.system("twine upload dist/*") os.system("make distclean") # ------------------------------------------------------ diff --git a/pyxie/model/cppnodes.py b/pyxie/model/cppnodes.py new file mode 100644 index 0000000..c4b0562 --- /dev/null +++ b/pyxie/model/cppnodes.py @@ -0,0 +1,678 @@ +# +# Copyright 2016 Michael Sparks +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +from __future__ import print_function +from __future__ import absolute_import + +from pyxie.model.iinodes import iiIdentifier + +import pyxie.codegen.simple_cpp +from pyxie.codegen.profiles import cpp_templates + +from pyxie.model.functions import builtins + +import pprint + +unique_id = 0 # FIXME: Naming - Used to generate unique ids within for statements. + # FIXME: Implies there should be a better way of doing this. + +def todo(*args): + print("TODO", " ".join([repr(x) for x in args])) + +def get_blank_line(): + return pyxie.codegen.simple_cpp.get_blank_line() + +def Print(*args): + return pyxie.codegen.simple_cpp.Print(*args) + +class CppNode(object): + pass + +# -------------------------------------------------------------------------------------------- +# Support code to cover intermediate stages of conversion +# +def node_type(node): + try: + nodeType = node.tag + except AttributeError as e: + print("*** WARNING ***") + print("WARNING: AttributeError:", e) + print("** WARNING ***") + nodeType = "UNDEFINED_NODETYPE" + + return nodeType + +def get_operator_type(arg): + return arg.operator + +def ExpressionIsPrintBuiltin(expression): + assert node_type(expression) == 'function_call' + + function_label = expression.iifunc_object + try: + function_name = function_label[1] + except TypeError as e: + if 'iiAttributeAccess' in str(e): + # This is an expected possibility + return False + if isIdentifier(function_label): + function_name = function_label.identifier + else: + raise + return function_name == "print" + +def isIdentifier(node): + return type(node) == iiIdentifier + +# END Support code to cover intermediate stages of conversion +# -------------------------------------------------------------------------------------------- + +def mkStatement(statement_spec): + ss = statement_spec + statement_type = node_type(statement_spec) + if statement_type == "assignment": + s = statement_spec + return CppAssignment( s.lvalue, s.rvalue, s.assignment_op) + + elif statement_type == "expression_statement": + expression = statement_spec.expression + if node_type(expression) == "function_call": + if ExpressionIsPrintBuiltin(expression): + print("XXXX", expression) + args = expression.iifunc_call_args + return CppPrintStatement(args) + return CppExpressionStatement(expression) + + elif statement_type == "while_statement": + return CppWhileStatement(statement_spec.condition, *statement_spec.statements) + + elif statement_type == "for_statement": + s = statement_spec + return CppForStatement(s.lvalue, + s.iterator, + s.statements, + s.for_statement_PyNode) + + elif statement_type == "if_statement": + if ss.extended_clause == None: + return CppIfStatement(ss.condition, ss.statements) + else: + return CppIfStatement(ss.condition, ss.statements, ss.extended_clause) + + elif statement_type == "pass_statement": + return CppEmptyStatement() + + elif statement_type == "break_statement": + return CppBreakStatement() + + elif statement_type == "continue_statement": + return CppContinueStatement() + + else: + print("Unknown statement type", statement_type, ss) + raise Exception("Unhandlable statement type: ", statement_type) + + + +class CppProgram(CppNode): + def __init__(self): + self.includes = [] + self.main_cframe = CppFrame() + self.name = "" + + @classmethod + def fromiinodes(klass, iiprogram): + program = klass() + program.name = iiprogram.name + program.includes = iiprogram.includes + + for identifier in iiprogram.global_identifiers: + value_type = identifier.value_type + name = identifier.name + program.main_cframe.identifiers.append(CppIdentifier(value_type, name)) + + for statement in iiprogram.statements: + conc_statement = mkStatement(statement) + program.main_cframe.statements.append(conc_statement) + + return program + + def generate(self, profile = "default"): + print("BUILDING FOR PROFILE", profile) + frame_lines = self.main_cframe.concrete() + seen = {} + for include in self.includes: + if not seen.get(include, False): + # Only output each include once + Print( "#include "+ include ) + seen[include] = True + + print_def = cpp_templates.get(profile, cpp_templates.get("default")) + frame_text = "\n".join([" "+line for line in frame_lines]) + + Print(print_def % { "FRAME_TEXT": frame_text } ) + + def json(self): + return { "PROGRAM": {"name": self.name, + "includes" : self.includes, + "main": self.main_cframe.json()} + } + + +class CppFrame(CppNode): + def __init__(self): + self.identifiers = [] + self.statements = [] + + def json(self): + for y in self.statements: + Print(y) + return {"c_frame": {"identifiers" : [ x.json() for x in self.identifiers ], + "statements" : [y.json() for y in self.statements ] } + } + + def concrete(self): + block = [] + for identifier in self.identifiers: + decl_code = identifier.decl_code() + block.append(decl_code + ";") + + block.append(get_blank_line() ) + for statement in self.statements: + if statement: + code = statement.code() + if code is None: + print("STATEMENT IS None, WHY?", statement) + block.append(code + ";") + return block + + +class CppIdentifier(CppNode): + def __init__(self, ctype, name): + self.ctype = ctype + self.name = name + + def json(self): + return ["identifier", self.ctype, self.name ] + + def decl_code(self): + return self.ctype + " " + self.name + + +class CppAssignment(CppNode): + def __init__(self, lvalue, rvalue, assigntype="="): + self.lvalue = lvalue + self.rvalue = rvalue + self.assigntype = assigntype + print("CppAssignment.__init__ LVALUE", lvalue, type(lvalue), repr(lvalue)) + print("CppAssignment.__init__ RVALUE", rvalue, type(rvalue), repr(rvalue)) + + def json(self): + return ["assignment", self.lvalue, self.assigntype, self.rvalue ] + + def code(self): + if node_type(self.rvalue) == "function_call": + print("Aha!",self.rvalue) + crvalue = CppArgumentList(self.rvalue) + crvalue = crvalue.code() + print("AHA!", crvalue) + + elif node_type(self.rvalue) == "op": + crvalue = CppArgumentList(self.rvalue) + crvalue = crvalue.code() + else: + print("Are we here?") + crvalue = self.rvalue + print("Are we here?",crvalue) + + print("self.lvalue",self.lvalue) + print("self.assigntype",self.assigntype) + print("crvalue",type(crvalue), crvalue) + return self.lvalue + " "+self.assigntype+" " + crvalue + + +# A pass statement is not a commented out statement. +# +# By that this: +# while True: pass +# +# Is equivalent to while(true); +# As opposed to while(true); +# +# So that's why this returns ";" not "" +class CppEmptyStatement(CppNode): + def json(self): + return ["pass_statement"] + + def code(self): + return ";" + + +class CppBreakStatement(CppNode): + def json(self): + return ["break_statement"] + + def code(self): + return "break" + + +class CppContinueStatement(CppNode): + def json(self): + return ["continue_statement"] + + def code(self): + return "continue" + + +def c_repr_of_expression(expression): + try: + cexpression = expression[1] + except TypeError: + cexpression = expression.identifier + return cexpression + + +class CppFunctionCall(CppNode): + def __init__(self, identifier, args): + self.args = args + self.identifier = identifier + self.arg_list = CppArgumentList(*args) + + def json(self): + return ["function_call", self.identifier ] + self.arg_list.json() + + def code(self): + if node_type(self.identifier) == "attributeaccess": + expression = self.identifier.expression + attribute = self.identifier.attribute + + c_expression = "(" + c_repr_of_expression(expression) + ")" + c_attribute = c_repr_of_expression(attribute) +# c_attribute = attribute[1] + + identifier = c_expression + "." + c_attribute + + else: + identifier = c_repr_of_expression(self.identifier) +# identifier = self.identifier[1] + + # FUDGE: The following would need to be profile specific + # + # It probably also needs reviewing/revising later on + if identifier == "print": + X = CppPrintStatement(*self.args) + return X.code() + + arglist = ", ".join(self.arg_list.code_list()) + return identifier + "(" + arglist + ")" + + + +class CppArgumentList(CppNode): + def __init__(self, *args): + self.args = args + + def json(self): + return list(self.args[:]) + + def code_op_F(self, arg): + if get_operator_type(arg) == "plus": return "+" + if get_operator_type(arg) == "minus": return "-" + if get_operator_type(arg) == "times": return "*" + if get_operator_type(arg) == "divide": return "/" + if get_operator_type(arg) == "boolean_or": return " || " + if get_operator_type(arg) == "boolean_and": return " && " + if get_operator_type(arg) == "boolean_not": return " ! " + + if get_operator_type(arg) in ["<", ">", "==", ">=", "<=", "!="]: + return get_operator_type(arg) + + if get_operator_type(arg) == "<>": return "!=" + + return None + + def code_op(self,arg): + c_op = self.code_op_F(arg) + if c_op: + if len(arg.args) == 2: + print("GENERATING CODE FOR INFIX OPERATORS") + # We would like to try to assert here that the values on both sides + # are integers, but at present we cannot do that, since we use too simplistic + # a structure. If I remove that constraint, we can generate more code. + # But we will need to revisit this. + lit_args = [ self.code_arg(x) for x in arg.args ] + result = "(" + lit_args[0] + c_op + lit_args[1] + ")" + return result + + if len(arg.args) == 1: + print("HERE INSTEAD") + arg1 = arg.args[0] +# # We will need to revisit this. + lit_arg1 = self.code_arg(arg1) + + result = "(" + c_op + lit_arg1 + ")" + return result + + todo("Handle code ops for anything other than plus/int,int") + raise NotImplementedError("Handle code ops for anything other than plus/int,int" + repr(arg)) + + + def code_arg(self, arg): + if node_type(arg) == "identifier": + return arg.identifier + elif node_type(arg) == "integer": + the_integer = arg.the_integer + r = repr(the_integer) + if the_integer<0: + r = "(" + r + ")" + return r + elif node_type(arg) == "string": + the_string = arg.the_string + carg = the_string.replace('"', '\\"') + return '"' + carg + '"' # Force double quotes + elif node_type(arg) == "double": + the_float = arg.the_float + return repr(the_float) + elif node_type(arg) == "boolean": + the_boolean = arg.the_boolean + return the_boolean + elif node_type(arg) == "op": + return self.code_op(arg) + elif node_type(arg) == "function_call": + print("WE REACH HERE") + code_gen = CppFunctionCall(arg.iifunc_object, arg.iifunc_call_args) + return code_gen.code() + print("We don't know how to generate code for function calls yet", arg) + return "" + + todo("Handle print value types that are more than the basic types", arg[0]) + raise NotImplementedError("Handle print value types that are more than the basic types" + repr(arg)) + + def code_list(self): + cargs = [] + for arg in self.args: + c_str = self.code_arg(arg) + cargs.append(c_str) + + return cargs + + def code(self): + return ",".join(self.code_list()) + + +class CppExpressionStatement(CppNode): + def __init__(self, expression): + self.expression = expression + + def json(self): + return ["expression_statement", self.expression ] + + def code(self): + print(self.expression) + if node_type(self.expression) == "function_call": + cvalue = CppArgumentList(self.expression) + cvalue = cvalue.code() + else: + raise Exception("Fail to understand how to generate code for %s" % repr(self.expression) ) + + return cvalue + + +class CppPrintStatement(CppNode): + def __init__(self, args): + self.args = args + self.arg_list = CppArgumentList(*args) + + def json(self): + return ["print_statement" ] + self.arg_list.json() + + def code(self): + return "( std::cout << " + " << \" \" << ".join(self.arg_list.code_list()) + " << std::endl )" + + +class CppWhileStatement(CppNode): + def __init__(self, condition, *statements): + self.raw_condition = condition + self.raw_statements = list(statements) + self.block_cframe = CppFrame() + + for statement in self.raw_statements: + conc_statement = mkStatement(statement) + self.block_cframe.statements.append(conc_statement) + + def json(self): + return ["while_statement", self.raw_condition] + self.raw_statements + + def code(self): + code = "while" + code += "(" + code += CppArgumentList(self.raw_condition).code() + code += ") {" + code += "\n".join( self.block_cframe.concrete() ) + code += "}" + return code + + +class CppForStatement(CppNode): + def __init__(self, identifier, iterable, statements, for_statement_pynode): + self.raw_identifier = identifier + self.raw_iterable = iterable + self.raw_statements = list(statements) + self.block_cframe = CppFrame() + self.for_statement_pynode = for_statement_pynode + + for statement in self.raw_statements: + conc_statement = mkStatement(statement) + self.block_cframe.statements.append(conc_statement) + + def json(self): + return ["for_statement", self.raw_condition, self.raw_iterable, self.raw_statements ] + + def code(self): + # + # The following is the sort of code we want to generate inline. + # This actually implements what we're after. + # However it requires some declarations to be in place really of the the find variables kind + # That said, C++ allows inline delcarations like this (for good or ill), so maybe just do + # this in the first instance, and leave improvements for refactoring??? + # + # Actually this discussion is useful, but it turns out to allow nested generator usage + # ie for x in range(5): for y in range(x): print x,y + # It requires this to be done closer to what could be viewed as "properly" + # +# %(ITERATOR_TYPE)s %(ITERATOR_NAME)s = %(ITERATOR_EXPRESSION)s; + template = """ + %(ITERATOR_NAME)s = %(ITERATOR_EXPRESSION)s; + while (true) { + %(IDENTIFIER)s = %(ITERATOR_NAME)s.next(); + if (%(ITERATOR_NAME)s.completed()) + break; + + %(BODY)s // Itself uses %(IDENTIFIER)s + } + """ + global unique_id + unique_id = unique_id+1 + + iterator_ctype = "range" + iterator_expression = self.raw_iterable ## NOTE THIS RESULTS IN ['iterate_over', ['function_call', ['identifier', 'range'], [['integer', 5]]]]- so that needs work + + print("OK, USE THIS THEN", self.for_statement_pynode.expression.ivalue_name) + iterator_name = self.for_statement_pynode.expression.ivalue_name + + if node_type(self.raw_iterable) == "iterator": + iterable = self.raw_iterable.expression + + if node_type(iterable) == "function_call": + print("FUNCTION_CALL", iterable.iifunc_object) + assert isIdentifier(iterable.iifunc_object) + + identifier = iterable.iifunc_object.identifier + if identifier in builtins: + print("YAY") + pprint.pprint( builtins[identifier] ) + iterator_ctype = builtins[identifier]["iterator_ctype"] + print("iterator_name -- ", iterator_ctype) + print("YAY") + + print(iterable) + fcall = CppFunctionCall(iterable.iifunc_object,iterable.iifunc_call_args) + fcall_code = fcall.code() + iterator_expression = fcall_code + + template_args = { "ITERATOR_TYPE": iterator_ctype, + "ITERATOR_EXPRESSION": iterator_expression, + "ITERATOR_NAME": iterator_name, + "UNIQIFIER":repr(unique_id), + "IDENTIFIER": self.raw_identifier, + "BODY": "\n".join( self.block_cframe.concrete() ) + } + + + + print("=== template args =================================================") + pprint.pprint (template_args ) + print("--- template args -------------------------------------------------") + if self.for_statement_pynode.expression.isiterator: + identifier = self.for_statement_pynode.identifier + print(identifier.get_type()) + print("=== end extractable template args =================================") + print(template % template_args) + + print("----------------------------------------------------------") + print(dir(self.for_statement_pynode)) + print(repr(self.for_statement_pynode)) + print("----------------------------------------------------------") +# raise NotImplementedError("Haven't finished implementing CppForStatement yet...") + code = template % template_args + + print("CODE", code) + + return code + + +class CppIfStatement(CppNode): + def __init__(self, condition, statements, extended_clause=None): + self.raw_condition = condition + self.raw_statements = list(statements) + self.extended_clause = extended_clause + + self.block_cframe = CppFrame() + for statement in self.raw_statements: + conc_statement = mkStatement(statement) + self.block_cframe.statements.append(conc_statement) + + def json(self): + result = ["if_statement", self.raw_condition, self.raw_statements ] + if self.extended_clause is not None: + result.append(extended_clauses.json()) + + def code(self): + extended_clauses_code = None + condition_code = CppArgumentList(self.raw_condition).code() + block_code = "\n".join( self.block_cframe.concrete() ) + if self.extended_clause: + if node_type(self.extended_clause) == "elif_clause": + condition = self.extended_clause.condition + statements = self.extended_clause.statements + if self.extended_clause.extended_clause: + extended_sub_clause = self.extended_clause.extended_clause + extended_clauses_code = CppElseIfClause( condition, statements, extended_sub_clause ).code() + else: + extended_clauses_code = CppElseIfClause( condition, statements ).code() + + if node_type(self.extended_clause) == "else_clause": + print("***************************************************") + statements = self.extended_clause.statements + extended_clauses_code = CppElseClause( statements ).code() + + code = "if ( %s ) { %s } " % (condition_code, block_code ) + if extended_clauses_code: + code = code + extended_clauses_code + return code + + +class CppElseIfClause(CppNode): + def __init__(self, condition, statements, extended_clause=None): + self.raw_condition = condition + self.raw_statements = list(statements) + self.extended_clause = extended_clause + + self.block_cframe = CppFrame() + for statement in self.raw_statements: + conc_statement = mkStatement(statement) + self.block_cframe.statements.append(conc_statement) + + def json(self): + result = ["elif_clause", self.raw_condition, self.raw_statements ] + if self.extended_clause is not None: + result.append(extended_clause.json()) + + def code(self): + extended_clauses_code = None + condition_code = CppArgumentList(self.raw_condition).code() + block_code = "\n".join( self.block_cframe.concrete() ) + if self.extended_clause: + if node_type(self.extended_clause) == "elif_clause": + print("***************************************************") + condition = self.extended_clause.condition + statements = self.extended_clause.statements + + if self.extended_clause.extended_clause: + extended_sub_clause = self.extended_clause.extended_clause + extended_clauses_code = CppElseIfClause( condition, statements, extended_sub_clause ).code() + else: + extended_clauses_code = CppElseIfClause( condition, statements ).code() + + + if node_type(self.extended_clause) == "else_clause": + print("***************************************************") + statements = self.extended_clause.statements + extended_clauses_code = CppElseClause( statements ).code() + + code = "else if ( %s ) { %s } " % (condition_code, block_code ) + if extended_clauses_code: + code = code + extended_clauses_code + return code + + +class CppElseClause(CppNode): + def __init__(self, statements): + self.raw_statements = list(statements) + + self.block_cframe = CppFrame() + for statement in self.raw_statements: + conc_statement = mkStatement(statement) + self.block_cframe.statements.append(conc_statement) + + def json(self): + result = ["elif_clause", self.raw_statements ] + + def code(self): + extended_clauses_code = None + block_code = "\n".join( self.block_cframe.concrete() ) + + code = "else { %s } " % (block_code, ) + + return code + + +#__all__ = [ "CppNode", "CppProgram", "CppFrame", "CppIdentifier", + #"CppAssignment", "CppEmptyStatement", "CppBreakStatement", + #"CppFunctionCall", "CppArgumentList", "CppExpressionStatement", + #"CppPrintStatement", "CppWhileStatement", "CppForStatement", + #"CppIfStatement", "CppElseIfClause", "CppElseClause" + #] + diff --git a/pyxie/model/iinodes.py b/pyxie/model/iinodes.py new file mode 100644 index 0000000..094853f --- /dev/null +++ b/pyxie/model/iinodes.py @@ -0,0 +1,299 @@ +# +# Copyright 2017 Michael Sparks +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +""" +This file will contain objects used to represent the independent intermediate +format of the program. + +Initially it's being a bridging/extraction point. +""" + +class iiNode(object): # This is to allow intermediate thunk check whether it has an iiNode or not... + tag = "iinode" + def __init__(self): + raise TypeError("Abstract Base class method called") + def __json__(self): + raise TypeError("Abstract Base class method called") + +class iiProgram(iiNode): + tag = "program" + def __init__(self, name, includes, identifiers, statements): + self.name = name + self.includes = includes[:] # Create shallow clone of includes + self.global_identifiers = identifiers + self.statements = statements + + def __json__(self): + # Backwards compatibility thunk - later will be used for debugging/a pretty printing representation + program = {} + program["name"] = self.name + program["includes"] = sorted(self.includes) + program["main"] = {} + + program["main"]["c_frame"] = {} + program["main"]["c_frame"]["identifiers"] = self.global_identifiers + program["main"]["c_frame"]["statements"] = self.statements + print("PROGRAM", program) + result = {"PROGRAM" : program } + return result + + +class iiAssignment(iiNode): + tag = "assignment" + def __init__(self, lvalue, assignment_op, rvalue): + self.lvalue = lvalue + self.assignment_op = assignment_op + self.rvalue= rvalue + def __json__(self): + return ["assignment", self.lvalue, self.assignment_op, self.rvalue ] + + +class iiOperator(iiNode): + tag = "operator" + def __init__(self, operator, args): + # FIXME: This is a transform that should happen outside this file... + if operator=="op_plus": + self.operator = "plus" + + elif operator=="op_minus": + self.operator = "minus" + + elif operator=="op_times": + self.operator = "times" + + elif operator=="op_divide": + self.operator = "divide" + + elif operator=="or_operator": + self.operator = "boolean_or" + + elif operator=="and_operator": + self.operator = "boolean_and" + + elif operator=="not_operator": + self.operator = "boolean_not" + + else: + raise ValueError("Cannot represent operator", (operator, args) ) + + self.tag = "op" + +# self.operator = operator + self.args = args + + + def __json__(self): + return ["operator", self.operator, self.args ] + + +class iiFunctionCall(iiNode): + tag = "function_call" + def __init__(self, func_object, args): + self.iifunc_object = func_object + self.iifunc_call_args = args + + def __json__(self): + return ["function_call", self.iifunc_object, self.iifunc_call_args ] + + +class iiAttributeAccess(iiNode): + tag = "attributeaccess" + def __init__(self, expression, attribute): + self.expression = expression + self.attribute = attribute + + def __json__(self): + return ["attributeaccess", self.expression, self.attribute] + +class iiIdentifier(iiNode): + tag = "identifier" + def __init__(self, identifier): + self.identifier = identifier + + def __json__(self): + return ["identifier", self.identifier] + + +class iiString(iiNode): + tag = "string" + def __init__(self, the_string): + self.the_string = the_string + + def __json__(self): + return ["string", self.the_string] + + +class iiInteger(iiNode): + tag = "integer" + def __init__(self, the_integer): + self.the_integer = the_integer + + def __json__(self): + return ["integer", self.the_integer] + +class iiFloat(iiNode): + tag = "double" + def __init__(self, the_float): + self.the_float = the_float + + def __json__(self): + return ["double", self.the_float] + + + +class iiBoolean(iiNode): + tag = "boolean" + def __init__(self, the_boolean): + self.the_boolean = the_boolean + + def __json__(self): + return ["boolean", self.the_boolean] + + +class iiComparison(iiNode): + tag = "op" + def __init__(self, comparator, left, right): + self.comparator = comparator # FIXME: Do we still need comparator? It's just a special case operator, right? + self.operator = comparator + self.args = [ left, right ] + self.left = left + self.right = right + def __json__(self): + return ["op", self.comparator, self.left, self.right] + + + +class iiWhileStatement(iiNode): + tag = "while_statement" + def __init__(self, condition, statements): + self.condition = condition + self.statements = statements + + def __json__(self): + return ["while_statement", self.condition] + self.statements + + +class iiIterator(iiNode): + tag = "iterator" + def __init__(self, expression): + self.expression = expression + + def __json__(self): + return ["iterator", self.expression] + + +class iiForStatement(iiNode): + tag = "for_statement" + def __init__(self, lvalue, iterator, statements, for_statement_PyNode): + self.lvalue = lvalue + self.iterator = iterator + self.statements = statements + self.for_statement_PyNode = for_statement_PyNode + + def __json__(self): + return ["for_statement", self.lvalue, self.iterator, self.statements, self.for_statement_PyNode] + + +class iiDefStatement(iiNode): + tag = "func_defintion" + def __init__(self, name, params, block, def_statement_PyNode): + self.name = name + self.params = params + self.block = block + self.def_statement_PyNode = def_statement_PyNode + + def __json__(self): + return ["func_defintion", self.name, self.params, self.block, repr(self.def_statement_PyNode) ] + + +class iiPassStatement(iiNode): + tag = "pass_statement" + def __init__(self): + pass + + def __json__(self): + return ["pass_statement"] + + +class iiBreakStatement(iiNode): + tag = "break_statement" + def __init__(self): + pass + def __json__(self): + return ["break_statement"] + +class iiContinueStatement(iiNode): + tag = "continue_statement" + def __init__(self): + pass + + def __json__(self): + return ["continue_statement"] + +class iiExpressionStatement(iiNode): + tag = "expression_statement" + def __init__(self, expression): + self.expression = expression + + def __json__(self): + return ["expression_statement", self.expression] + + + +class iiIdentifierDeclaration(iiNode): + tag = "identifier" + def __init__(self, name, value_type): + self.value_type = value_type + self.name = name + + def __json__(self): + return ["identifier", self.value_type, self.name] + +class iiElifClause(iiNode): + tag = "elif_clause" + def __init__(self, condition, statements, extended_clause=None): + self.condition = condition + self.statements = statements + self.extended_clause = extended_clause + + def __json__(self): + if self.extended_clause: + return ["elif_clause", self.condition, self.statements, self.extended_clause] + else: + return ["elif_clause", self.condition, self.statements] + + +class iiElseClause(iiNode): + tag = "else_clause" + def __init__(self, statements ): + self.statements = statements + + def __json__(self): + return ["else_clause", self.statements] + + +class iiIfStatement(iiNode): + tag = "if_statement" + def __init__(self, condition, statements, extended_clause=None): + self.condition = condition + self.statements = statements + self.extended_clause = extended_clause + + def __json__(self): + if self.extended_clause: + return ["if_statement", self.condition, self.statements, self.extended_clause] + else: + return ["if_statement", self.condition, self.statements] + diff --git a/pyxie/model/pynodes/operators.py b/pyxie/model/pynodes/operators.py index 0c9e756..49180f7 100644 --- a/pyxie/model/pynodes/operators.py +++ b/pyxie/model/pynodes/operators.py @@ -80,6 +80,9 @@ def analyse(self): self.ntype = self.get_type() + def args(self): + return [ self.arg1, self.arg2 ] + class PyBoolOperator(PyOperation): tag = "base_bool_operator" @@ -105,6 +108,9 @@ def __info__(self): i = i + 1 return info + def args(self): + return list(self.argv) + def __repr__(self): return "%s%s" % (self.classname(),repr(tuple([x for x in self.argv]))) @@ -178,6 +184,9 @@ def type(self): def __repr__(self): return "%s(%s, %s, %s)" % (self.classname(),self.comparison, repr(self.arg1),repr(self.arg2)) + def args(self): + return [ self.arg1, self.arg2 ] + def __json__(self): return [ self.tag, self.comparison, jdump(self.arg1), jdump(self.arg2) ] diff --git a/pyxie/model/transform.py b/pyxie/model/transform.py index 1d1ffe3..1c755ea 100755 --- a/pyxie/model/transform.py +++ b/pyxie/model/transform.py @@ -1,6 +1,6 @@ #!/usr/bin/python # -# Copyright 2016 Michael Sparks +# Copyright 2018 Michael Sparks # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -29,6 +29,8 @@ from pyxie.model.functions import builtins from pyxie.model.functions import profile_types +from pyxie.model.iinodes import * + iterator_unique_base = 0 class UnknownType(Exception): @@ -37,6 +39,16 @@ class UnknownType(Exception): class CannotConvert(Exception): pass +# +# Support code to cover intermediate stages of conversion +# +def statement_type(cstatement): + try: + statementType = cstatement[0] + except TypeError: + statementType = cstatement.tag + return statementType + def todo(*args): @@ -116,8 +128,9 @@ def python_type_to_c_type(ptype): def includes_for_ctype(ctype): if ctype == "string": return "" + def includes_for_cstatement(cstatement): - if cstatement[0] == "print_statement": return "" + if statement_type(cstatement) == "print_statement": return "" def crepr_literal(pyliteral): assert isinstance(pyliteral, nodes.PyValueLiteral) @@ -157,25 +170,16 @@ def crepr_literal(pyliteral): def crepr_op(py_op): assert isinstance(py_op, nodes.PyOperator) or isinstance(py_op, nodes.PyBoolOperator) - func = py_op.tag - - if func == "op_plus": - return ["op", "plus"] - if func == "op_minus": - return ["op", "minus"] - if func == "op_times": - return ["op", "times"] - if func == "op_divide": - return ["op", "divide"] - if func == "or_operator": - return ["op", "boolean_or"] - if func == "and_operator": - return ["op", "boolean_and"] - if func == "not_operator": - return ["op", "boolean_not"] - else: - todo("Cannot yet convert operators functions other than plus...") - raise CannotConvert("Cannot yet convert operators functions other than plus...:" + repr(py_op)) + + args = py_op.args() + crepr_args = [ convert_arg(x) for x in args ] + + try: + result = iiOperator(py_op.tag, crepr_args) + except ValueError: + raise CannotConvert("Cannot represent operator", py_op.tag) + return result + def convert_assignment(assignment): lvalue, assign_type, rvalue = assignment.lvalue, assignment.assign_type, assignment.rvalue @@ -223,15 +227,16 @@ def convert_assignment(assignment): cargs = [] if arg.expr_list: for expr in arg.expr_list: - #print(arg) - #print("We need to convert the arg", arg) crepr = convert_arg(expr) carg = crepr cargs.append(carg) - crvalue = ["function_call", convert_value_literal(arg.func_label), cargs] # FIXME: func_label may not be an identifier... + func_label = convert_value_literal(arg.func_label) # FIXME: func_label may not be an identifier... + crvalue = iiFunctionCall(func_object=func_label, args=cargs) + + result = iiAssignment(lvalue=clvalue, assignment_op="=", rvalue=crvalue) + return result - return ["assignment", clvalue, "=", crvalue] def convert_value_literal(arg): # print(repr(arg), arg) @@ -241,7 +246,8 @@ def convert_value_literal(arg): if arg.tag == "attributeaccess": expression = convert_value_literal(arg.expression) attribute = convert_value_literal(arg.attribute) - return ["attributeaccess", expression, attribute] + + return iiAttributeAccess(expression, attribute) if arg.tag == "attribute": x = convert_value_literal(arg.value) @@ -250,20 +256,23 @@ def convert_value_literal(arg): tag, value, vtype, line = arg.tag, arg.value, arg.get_type(), arg.lineno if tag == "identifier": - return ["identifier", value] + return iiIdentifier(identifier=value) if vtype == "string": - return ["string", value] + return iiString(the_string=value) + if vtype == "integer": - return ["integer", value] + return iiInteger(the_integer=value) + if vtype == "float": - return ["double", value] + return iiFloat(the_float=value) + if vtype == "bool": if value == True: value = "true" else: value = "false" - return ["boolean", value] + return iiBoolean(the_boolean=value) todo("CONVERSION: Cannot handle other value literals %s" % repr(arg)) todo("CONVERSION: %s %s %s %d" % (tag, repr(value), repr(vtype), line)) @@ -291,11 +300,8 @@ def convert_bool_operator_function(opfunc): print("crepr_arg1", repr(crepr_arg1)) - if arg2: - print("crepr_arg2", repr(crepr_arg2)) - result = crepr_op(opfunc) + [crepr_arg1, crepr_arg2] - else: - result = crepr_op(opfunc) + [crepr_arg1] + result = crepr_op(opfunc) + print(repr(result)) return result @@ -313,13 +319,14 @@ def convert_operator_function(opfunc): print("crepr_arg1", repr(crepr_arg1)) print("crepr_arg2", repr(crepr_arg2)) - result = crepr_op(opfunc) + [crepr_arg1, crepr_arg2] + result = crepr_op(opfunc) print(repr(result)) return result #todo("Cannot yet convert operator functions") #raise CannotConvert("Cannot convert operator function :" + repr(arg)) + def convert_comparison_operator(comparison): # t_NORMAL_COMPARISON_OPERATOR = r'(in|not +in|is|is +not)' @@ -328,6 +335,7 @@ def convert_comparison_operator(comparison): raise NotImplementedError(repr(comparison)) + def convert_comparison(comparison_spec): print("CONVERT - convert_comparison", repr(comparison_spec)) assert isinstance(comparison_spec, nodes.PyComparisonOperator) @@ -342,12 +350,12 @@ def convert_comparison(comparison_spec): print("crepr_arg1", repr(crepr_arg1)) print("crepr_arg2", repr(crepr_arg2)) - result = ["op", crepr_comparison, crepr_arg1, crepr_arg2] + result = iiComparison(comparator=crepr_comparison, left=crepr_arg1, right=crepr_arg2) + print(repr(result)) return result - def convert_arg(arg): if isinstance(arg, nodes.PyValueLiteral): print("CONVERTING LITERAL", arg) @@ -370,10 +378,13 @@ def convert_arg(arg): carg = crepr cargs.append(carg) - return ["function_call", convert_value_literal(arg.func_label), cargs] # FIXME: func_label may not be an identifier... + func_label = convert_value_literal(arg.func_label) # FIXME: func_label may not be an identifier... + return iiFunctionCall(func_object=func_label, args=cargs) + else: todo("Handle print for non-value-literals") - raise CannotConvert("Cannot convert print for non-value-literals") + raise CannotConvert("Cannot convert print for non-value-literals", arg) + def convert_print(print_statement): cstatement = [] @@ -384,12 +395,15 @@ def convert_print(print_statement): crepr = convert_arg(arg) carg = crepr cargs.append(carg) - return ["print_statement"] + cargs + return mkPrintStatement(args=cargs) + def convert_while_statement(while_statement): crepr_condition = convert_arg(while_statement.condition) cstatements = convert_statements(while_statement.block) - return ["while_statement", crepr_condition] + cstatements + + return iiWhileStatement(condition=crepr_condition, statements=cstatements) + def convert_for_statement(for_statement): @@ -398,9 +412,11 @@ def convert_for_statement(for_statement): step = for_statement.block clvalue = lvalue.value # FIXME: This is only valid for identifiers - crvalue_source = ["iterator", convert_arg(rvalue_source)] - cstep = convert_statements(step) + crvalue_source = convert_arg(rvalue_source) + crvalue_source = iiIterator(expression=crvalue_source) + + cstep = convert_statements(step) print("*******************") print(crvalue_source) @@ -417,10 +433,9 @@ def convert_for_statement(for_statement): print("*******************") pprint.pprint(for_statement.__info__()) print("*******************") - # crepr_condition = convert_arg(for_statement.condition) - # cstatements = convert_statements(while_statement.block) -# return ["while_statement", crepr_condition] + cstatements - return ["for_statement", clvalue, crvalue_source, cstep, for_statement] + + return iiForStatement(lvalue=clvalue, iterator=crvalue_source, + statements=cstep, for_statement_PyNode=for_statement) def convert_def_statement(def_statement): @@ -433,9 +448,6 @@ def convert_def_statement(def_statement): print("BLOCK:", def_statement.block) print("*******************") - -# raise CannotConvert("Statement: def_statement") - lvalue = def_statement.identifier cidentifer = lvalue.value # FIXME: This is only valid for identifiers @@ -443,10 +455,7 @@ def convert_def_statement(def_statement): block = def_statement.block cblock = convert_statements(block) - - -# return ["func_defintion", cidentifer, cparamlist , cblock, def_statement] - return ["func_defintion", cidentifer, cparamlist , cblock, repr(def_statement) ] + return iiDefStatement(name=cidentifer, params=cparamlist, block=cblock, def_statement_PyNode=def_statement) def convert_extended_clause(extended_clause): @@ -456,13 +465,17 @@ def convert_extended_clause(extended_clause): cstatements = convert_statements(extended_clause.block) if extended_clause.else_clause: cextended_clause = convert_extended_clause(extended_clause.else_clause) - return["elif_clause", crepr_condition, cstatements, cextended_clause] - return ["elif_clause", crepr_condition, cstatements] + + return iiElifClause(condition=crepr_condition, statements=cstatements, + extended_clause=cextended_clause) + + return iiElifClause(condition=crepr_condition, statements=cstatements) if extended_clause.tag == "else_clause": print("WORKING THROUGH ELSE:", extended_clause) cstatements = convert_statements(extended_clause.block) - return ["else_clause", cstatements] + + return iiElseClause(statements=cstatements) print("NOT ELIF!") print("NOT ELSE!") @@ -476,17 +489,23 @@ def convert_if_statement(if_statement): cstatements = convert_statements(if_statement.block) if if_statement.else_clause: cextended_clause = convert_extended_clause(if_statement.else_clause) - return["if_statement", crepr_condition, cstatements, cextended_clause] - return ["if_statement", crepr_condition, cstatements] + + return iiIfStatement(condition=crepr_condition, statements=cstatements, extended_clause=cextended_clause) + + return iiIfStatement(condition=crepr_condition, statements=cstatements) + def convert_pass_statement(pass_statement): - return ["pass_statement"] + return iiPassStatement() + def convert_break_statement(break_statement): - return ["break_statement"] + return iiBreakStatement() + def convert_continue_statement(continue_statement): - return ["continue_statement"] + return iiContinueStatement() + def convert_expression_statement(statement): print("CONVERTING EXPRESSION STATEMENTS", statement.value) @@ -494,7 +513,8 @@ def convert_expression_statement(statement): crepr = convert_arg(statement.value) print("REACHED HERE") print("CONVERTED ", crepr) - return ["expression_statement", crepr] + + return iiExpressionStatement(expression=crepr) def convert_statements(AST): @@ -525,7 +545,8 @@ def convert_statements(AST): cstatement = convert_if_statement(statement) cstatements.append(cstatement) elif statement.tag == "pass_statement": - cstatement = convert_break_statement(statement) + cstatement = convert_pass_statement(statement) + cstatements.append(cstatement) elif statement.tag == "break_statement": cstatement = convert_break_statement(statement) cstatements.append(cstatement) @@ -546,18 +567,23 @@ def ast_to_cst(program_name, AST): ast_includes = [x.replace("#include ","") for x in AST.includes] print("AST.includes = ", ast_includes) + # Extract and handle variables + pvariables = find_variables(AST) cvariables = [] ctypes = {} + # includes = [] includes = [x.replace("#include ","") for x in AST.includes] names = pvariables.keys() names.sort() for name in names: ctype = python_type_to_c_type(pvariables[name]) - identifier = ["identifier", ctype, name] - cvariables.append(identifier) + + identifier_declaration = iiIdentifierDeclaration(name=name, value_type=ctype) + + cvariables.append(identifier_declaration) ctypes[ctype] = True print("cvariables",cvariables) @@ -574,19 +600,13 @@ def ast_to_cst(program_name, AST): print("INCLUDES::", includes) # Based on statements, update includes for cstatement in cstatements: + print( "GETTING INCLUDES", cstatement ) inc = includes_for_cstatement(cstatement) if inc: includes.append(inc) - program = {} - program["name"] = program_name - program["includes"] = sorted(includes) - program["main"] = {} - program["main"]["c_frame"] = {} - program["main"]["c_frame"]["identifiers"] = cvariables - program["main"]["c_frame"]["statements"] = cstatements - - return { "PROGRAM" : program } + program = iiProgram(name=program_name, includes=includes, identifiers=cvariables, statements=cstatements) + return program if __name__ == "__main__": diff --git a/pyxie/profile/arduino.py b/pyxie/profile/arduino.py index 42b0b4c..1433814 100644 --- a/pyxie/profile/arduino.py +++ b/pyxie/profile/arduino.py @@ -8,12 +8,16 @@ def initialise_external_function_definitions(): "Servo": { "iterator": False, "return_ctype": "Servo", # C type of the returned value + }, + "Adafruit_NeoPixel": { + "iterator": False, + "return_ctype": "Adafruit_NeoPixel", # C type of the returned value } } types = { - "Servo": { - } + "Servo": { }, + "Adafruit_NeoPixel": {} } # Update the actual profile functions/types @@ -42,6 +46,12 @@ def populate_profile_context(context): a_pin = ProfilePyNode("OUTPUT", "integer") context.store("OUTPUT", a_pin) + a_def= ProfilePyNode("NEO_GRB", "integer") + context.store("NEO_GRB", a_pin) + a_def = ProfilePyNode("NEO_KHZ800", "integer") + context.store("NEO_KHZ800", a_pin) + + def initialise_profile(context): context.tag = "PROFILE:arduino" diff --git a/setup.py b/setup.py index e59759f..ba622ee 100644 --- a/setup.py +++ b/setup.py @@ -18,8 +18,9 @@ # NOTE: EDIT site/setup.tmpl instead # -from distutils.core import setup -from distutils.version import LooseVersion +# from distutils.core import setup +from setuptools import setup + import os def is_package(path): @@ -48,7 +49,7 @@ def find_packages(path, base="" ): package_names = packages.keys() setup(name = "pyxie", - version = "0.1.24", + version = "0.1.26", description = "Little Python to C++ Compiler", url='http://www.sparkslabs.com/pyxie/', author='Michael Sparks (sparkslabs)', @@ -69,8 +70,8 @@ def find_packages(path, base="" ): What job does / will this do? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The aim of this project is to allow a adults and children to write code -in a familiar high level language that can then be compiled to run on an +The aim of this project is to allow adults and children to write code in +a familiar high level language that can then be compiled to run on an arbitrary embedded system - that is devices with very low power CPUs and very little memory. (ie devices too small to host a python interpreter/runtime) @@ -305,6 +306,9 @@ def find_packages(path, base="" ): Release History: +- 0.1.26 - UNRELEASED - TBD +- 0.1.25 - 2018-02-03 - Major changes to internal representation and + some new examples - 0.1.24 - 2016-11-10 - Fix assignment in else/elif, update to support Ubuntu 16.04LTS, initial steps on function/def statement support - 0.1.23 - 2016-10-11 - Major arduino profile improvements, @@ -496,7 +500,7 @@ def find_packages(path, base="" ): simplifying the syntax, it also means that Arudino statements like Serial.print now become legal statements. -Michael Sparks, November 2016 +Michael Sparks, February 2018 """ ) diff --git a/site/build_site_local.py b/site/build_site_local.py index bf40ca0..10e54cb 100644 --- a/site/build_site_local.py +++ b/site/build_site_local.py @@ -48,7 +48,7 @@ {% grammar = panel("panels/why-python-2-print.md") %} -Michael Sparks, MONTH 2016 +Michael Sparks, MONTH 2018 """ diff --git a/site/src/changelog.md b/site/src/changelog.md index b7cc689..b7ddcc7 100644 --- a/site/src/changelog.md +++ b/site/src/changelog.md @@ -3,14 +3,104 @@ template: mainpage source_form: markdown name: Changelog title: Changelog -updated: November 2016 +updated: February 2018 --- ## Change Log All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/), within reason. (Prior to a 1.0.0 release minor version increases may be backwards -incompatible) +incompatible) See doc/Versioning.md for more details + +## [0.1.26] - UNRELEASED + +### New + +* + +### What's been fixed? + +* + +### Internal Changes + +* + +### Other + +* + +## [0.1.25] - 2018-02-03 + +The focus of this release is the result of the independent intermediate +node refactoring project I was undertaking. This was to transform the list +oriented data structure for representing a "pure" intermediate form of +code into a more stuctured/generalisable format. + +The motivation behind this change was the realisation that implementation +of end user functions (ie `def`) would be very difficult with this more +structured approach. + +### New + +* `doc/Versioning.md` - semantic versioning as it applies to pyxie +* `doc/WIPNOTES/6.Models.md` - start of some docs around the models in use +* Added explicit notes on licensing of pyxie's output. + (Short version: I view the output as being derived from your code by you) +* Language focussed examples / acceptance tests added: + `if` `if-else` `pass` `print` `while-break` `while-continue` +* Change `arduino` profile to support the `Adafruit_NeoPixel` library +* `neopixel` example added + +### What's been fixed? / Improved + +* Handling if if/elif/else improved/fixed +* added `clean.sh` to `arduino` example +* added `clean.sh` to `servo` example +* added `README.md` to `simplest_function` example (won't work yet) + +### Internal Changes + +* `bin/pyxie` now pulls in functionality from `pyxie.api` to be clearer + about what the API is +* added `pyxie/api.py` - public API for embedding pyxie. (Assumes + ubuntu host) Core contents: + * `initialise_API(profile="default")` - Call first + * `set_profile(profile_name)` - Call later + * `PyxieAPI.parse(filename)` - parse file, output goes to console + * `PyxieAPI.analyse(filename)` - parse & analyse flle, output to console + * `PyxieAPI.codegen(filename, result_filename=None)` - parse through + code generation. Output to console + * `PyxieAPI.compile(filename, result_filename=None)` - Compile file, + result ends up as "result_filename" +* `pyxie/core` - changed over to drive the conversion of pynodes to + cppnodes via iinodes. (aim is to simplify pynodes down the line and + make iinodes our main location for analysis/etc) +* Minor changes to `.gitignore` +* Minor change to the `Makefile` to make editting simpler for me... +* Update `clib.py` based on changes to `clib` +* `pyxie/codegen/simple_cpp.py` - functionality shifted out to + `pyxie/models/cppnodes.py` to allow a model driven approach +* `pyxie/model/cppnodes.py` Created. A better code representation + model for C++. Code generation therefore means transforming from the + `iiNode`s to `CppNode`s +* `pyxie/model/iinodes.py` - Introduces `iiNode`s - which are used to + represent an intermediate version of a program such that it can bridge + the conversion between languages +* `pyxie/model/pynodes/operators.py` - added `args()` method to a number + of classes to unify behaviours +* `pyxie/model/transform.py` - Major change - converts `PyNode`s to + `iiNode` representation rather than json objects. + +### Other + +* `doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md` corrected release date +* `doc/newsletter/XX-TBD.Pyxie-2017-Goals.md` - Unreleased newsletter. + Interesting notes, will get reused if appropriate later. Rather moot + contents now though. +* `doc/newsletter/08-TBD.Focus-Or-Pyxie-Release-X.X.XX.md` Template for + next newsletter + ## [0.1.24] - 2016-11-10 @@ -73,7 +163,6 @@ incompatible) * Add special case for function calls like print #37 - ## [0.1.23] - 2016-10-11 Not backwards compatible due to changes to "print" diff --git a/site/src/dev-status.md b/site/src/dev-status.md index e70ca42..ff1d6e4 100644 --- a/site/src/dev-status.md +++ b/site/src/dev-status.md @@ -2,7 +2,7 @@ template: mainpage source_form: markdown name: Dev Status -updated: 11 October 2016 +updated: 08 February 2018 title: Detailed Dev status --- ## Detailed Development status diff --git a/site/src/index.md b/site/src/index.md index da975b7..666fd16 100644 --- a/site/src/index.md +++ b/site/src/index.md @@ -7,7 +7,7 @@ updated: October 2016 --- ## Pyxie -- A Little Python to C++ Compiler -**Latest Release:** [0.1.23](changelog.html) (12/Oct/2016) +**Latest Release:** [0.1.25](changelog.html) (3/Feb/2018) {% whatjobdoesthisdo = panel("panels/what-job-does-this-do.md") %} diff --git a/site/src/panels/shortlog.md b/site/src/panels/shortlog.md index 1238a30..9ecf1a4 100644 --- a/site/src/panels/shortlog.md +++ b/site/src/panels/shortlog.md @@ -5,6 +5,8 @@ name: Shortlog updated: August 2016 title: Shortlog --- +* 0.1.26 - UNRELEASED - TBD +* 0.1.25 - 2018-02-03 - Major changes to internal representation and some new examples * 0.1.24 - 2016-11-10 - Fix assignment in else/elif, update to support Ubuntu 16.04LTS, initial steps on function/def statement support * 0.1.23 - 2016-10-11 - Major arduino profile improvements, print-as-function not statement * 0.0.22 - 2016-09-25 - Enable ability to use a variety of Arduino boards by using an Makefile.in file diff --git a/site/src/panels/trello-cardlist-backlog-arising.md b/site/src/panels/trello-cardlist-backlog-arising.md index a2a5a0e..563ee37 100644 --- a/site/src/panels/trello-cardlist-backlog-arising.md +++ b/site/src/panels/trello-cardlist-backlog-arising.md @@ -2,9 +2,11 @@ template: mainpanel source_form: markdown name: Trello Cardlist Backlog: Arising -updated: October 2016 +updated: February 2018 title: Trello Cardlist Backlog: Arising --- ### Backlog: Arising -* 259\. Would be nice to remove extraneous brackets round generated C++ expressions +* 292\. Debian packaging should include examples +* 294\. Website should include examples. +* 295\. Move analysis to pyxie.transform.analysis diff --git a/site/src/panels/trello-cardlist-backlog-features.md b/site/src/panels/trello-cardlist-backlog-features.md index 5e75ec9..089fea2 100644 --- a/site/src/panels/trello-cardlist-backlog-features.md +++ b/site/src/panels/trello-cardlist-backlog-features.md @@ -2,51 +2,52 @@ template: mainpanel source_form: markdown name: Trello Cardlist Backlog: Features -updated: October 2016 +updated: February 2018 title: Trello Cardlist Backlog: Features --- ### Backlog: Features -* 251\. Profile Context is usable for defining externally defined methods/functions (for typechecking) -* 9\. Add the manpage to the distributed files, into the right places #docs -* 8\. Add updating man page to the makefile #docs -* 197\. Web Editor +* 31\. Features focusses on BARE level functionality +* ----------------------------- +* == NEED ====== +* 38\. Generalise special casing of function calls. #practicalities +* 40\. Core expression code generation for strings #pylang +* 41\. Implement container oriented comparison operators #pylang +* 42\. Implement identity oriented comparison operators #pylang +* 48\. Modulo Operator for Integers #pylang +* ----------------------------- +* == USEFUL ====== +* 33\. Can find_variables(AST) in pyxie.model.transform actually just look in the results of the analysis phase? It should be able to. Building the same structure after all #internals +* 36\. Comments are implemented - would be useful for documented tests more appropriately #pylang +* 28\. Better Error messages for users #pylang #practicalties +* 14\. Code Cleanups #refactor #internals * 4\. Compilation profiles are pluggable -* 11\. MBed compatible compilation profile? (Seeedstudio Arch) -* 12\. MSP430 compatible compilation profile? * 13\. Pyxie compile harness supports custom working directories #practicalities -* 14\. Code Cleanups #refactor #internals -* 15\. Refactor code generation #refactor #internals -* 16\. Find Variables duplicates effort from the analysis phase, while also relying on results from it #internals -* 17\. Transform step is throwing away data. This seems broken and should perhaps pass through decorated pynodes - or at least retain a reference. #internals -* 18\. Code generation of C literals is muddled up a touch with structural representation #internals -* 19\. Website should use responsive CSS #website -* 20\. Website side bar could be better implemented. #website * 21\. Truthiness for values that AND/OR/NOT arguments needs resolving properly in C. (deferred) #pylang * 22\. Truthiness of expressions is explicitly checked/valid - for us in if and while statements and boolean expressions #pylang #internals -* 23\. Website could do with some pictures :-) #website +* 47\. Modulo Operator for Strings #pylang +* ----------------------------- +* == WANT ====== +* Flesh out a micro:bit profile. +* 196\. Batch Compiler +* 197\. Web Editor +* 39\. Variables inside while loops are handled correctly #testing +* ----------------------------- +* == WOULDLIKE ====== * 24\. Currently have 2 shift/reduce conflicts. They're auto-resolved correctly, but could be worth seeing if they could be resolved better. #pylang #internals +* 59\. Start thoughts on pyxie-web #reflect #website +* 9\. Add the manpage to the distributed files, into the right places #docs +* 8\. Add updating man page to the makefile #docs +* 11\. MBed compatible compilation profile? (Seeedstudio Arch) +* 12\. MSP430 compatible compilation profile? * 25\. Consider using CppHeaderParser on the C++ side of things - to inform the code generation side of things #internals #pylang -* 26\. Block structure of generated C Code is pretty/human friendly/readable #internals +* 259\. Would be nice to remove extraneous brackets round generated C++ expressions * 27\. Operations and operators could be unified with a bit of tweaking - using "x.operation", not "x.tag/x.comparison" etc #internals #refactor -* 28\. Better Error messages for users #pylang #practicalties * 29\. Duplication exists within code generation for operators. (cf convert_operator etc) #internals #refactor -* 31\. Features focusses on BARE level functionality -* 32\. C Syntax Tree is a Tree #internals -* 33\. Can find_variables(AST) in pyxie.model.transform actually just look in the results of the analysis phase? It should be able to. Building the same structure after all #internals -* 34\. Function call code supports simplified type definitions for externals. #practicalities * 35\. Unify Boolean operator pynode implementations #refactor #internals -* 36\. Comments are implemented - would be useful for documented tests more appropriately #pylang -* 37\. Add special case for function calls like print... #pylang -* 38\. Generalise special casing of function calls. #practicalities -* 39\. Variables inside while loops are handled correctly #testing -* 40\. Core expression code generation for strings #pylang -* 41\. Implement container oriented comparison operators #pylang -* 42\. Implement identity oriented comparison operators #pylang * 43\. Code generation for integer literals retains original representation where possible (Octal, hex, binary, etc) #practicalities * 44\. Function call code supports C style function prototypes for type definitions #tbd * 45\. Function call code supports name C headers for type definitions #tbd * 46\. Pyxie caches results of C style header parsing #tbd -* 47\. Modulo Operator for Strings #pylang -* 48\. Modulo Operator for Integers #pylang * 49\. Modulo Operator for Floats #pylang +* == NOT PRIORITISED ====== diff --git a/site/src/panels/trello-cardlist-backlog-proposed-next.md b/site/src/panels/trello-cardlist-backlog-proposed-next.md index 77d6e92..f736fde 100644 --- a/site/src/panels/trello-cardlist-backlog-proposed-next.md +++ b/site/src/panels/trello-cardlist-backlog-proposed-next.md @@ -2,19 +2,22 @@ template: mainpanel source_form: markdown name: Trello Cardlist Backlog: Proposed Next -updated: October 2016 +updated: February 2018 title: Trello Cardlist Backlog: Proposed Next --- ### Backlog: Proposed Next -* 196\. Batch Compiler -* 59\. Start thoughts on pyxie-web #reflect #website -* 195\. Functionality of bin/pyxie-dev is in core, not a script -* 3\.5 Analysis code looks for an arduino profile file describing c-types appropriately. -* 3\.6 arduino profile file is propogated with something relating to core functions that read values, to avoid forcing types "manually" -* 213\. v0 no funcs Playful Puppy code analyses -* 214\. v0 no funcs Playful Puppy code generates code -* 215\. v0 no funcs Playful Puppy code compiles, and runs on device correctly +* 271\. Transformation from python AST to C-CST exists/works - for simplest possible function +* 268\. Code generation for very basic functions succeeds. (no args, return values or variables) +* 267\. Analysis of functions not using any variables succeeds (no args, no return values) * 221\. Initial spike support for function definitions. (no args, no return values) +* 273\. Code generation for functions has cross overs for that of identifier definition. +* ----------------------------- * 223\. Spike support for functions with basic arguments. * 222\. Spike support for functions which use local variables +* 251\. Profile Context is usable for defining externally defined methods (for typechecking) +* 227\. .get_type() should be delegated, not rely on internal pynode details. +* 15\. Refactor code generation #refactor #internals +* 16\. Find Variables duplicates effort from the analysis phase, while also relying on results from it #internals +* 17\. Transform step is throwing away data. This seems broken and should perhaps pass through decorated pynodes - or at least retain a reference. #internals +* 58\. Blog Post: Pyxie Decisions #website diff --git a/site/src/panels/trello-cardlist-backlog-tasks.md b/site/src/panels/trello-cardlist-backlog-tasks.md index 2f010f7..dde9d3a 100644 --- a/site/src/panels/trello-cardlist-backlog-tasks.md +++ b/site/src/panels/trello-cardlist-backlog-tasks.md @@ -2,12 +2,10 @@ template: mainpanel source_form: markdown name: Trello Cardlist Backlog: Tasks -updated: October 2016 +updated: February 2018 title: Trello Cardlist Backlog: Tasks --- ### Backlog: Tasks * 52\. Review whether context should check types of ALL expressions, rather than just first, and whether we can should try to detect type mismatches #reflect -* 53\. Use https://travis-ci.org/ ? #reflect -* 54\. Review pypi packaging for things we should be doing #reflect #practicalities -* 55\. Create pyxie-service for managing batch compilation services #practicalities +* 272\. Review Core Python Types To Do Next diff --git a/site/src/panels/trello-cardlist-backlog-website-docs-etc.md b/site/src/panels/trello-cardlist-backlog-website-docs-etc.md index 0d7e9bb..aeba3aa 100644 --- a/site/src/panels/trello-cardlist-backlog-website-docs-etc.md +++ b/site/src/panels/trello-cardlist-backlog-website-docs-etc.md @@ -2,14 +2,16 @@ template: mainpanel source_form: markdown name: Trello Cardlist Backlog: Website, docs, etc -updated: October 2016 +updated: February 2018 title: Trello Cardlist Backlog: Website, docs, etc --- ### Backlog: Website, docs, etc +* 20\. Website side bar could be better implemented. #website +* 50\. Link current test programs on the website, maybe #website * 232\. Would be nice to have a prettier website +* 23\. Website could do with some pictures :-) #website * 233\. Some sort of Logo would be nice -* 58\. Blog Post: Pyxie Decisions #website * 60\. Start thoughts on pyxie-gui #reflect * 30\. Should we allow comments on website? #website -* 50\. Link current test programs on the website, maybe #website +* 19\. Website should use responsive CSS #website diff --git a/site/src/panels/trello-cardlist-features-implemented.md b/site/src/panels/trello-cardlist-features-implemented.md index ecc0f70..3c9c12d 100644 --- a/site/src/panels/trello-cardlist-features-implemented.md +++ b/site/src/panels/trello-cardlist-features-implemented.md @@ -2,11 +2,46 @@ template: mainpanel source_form: markdown name: Trello Cardlist Features Implemented -updated: October 2016 +updated: February 2018 title: Trello Cardlist Features Implemented --- ### Features Implemented +* 296\. Remove a large chunk of old extraneous debugging output +* 290\. Rename "codegen" as "transform" +* 0\.1.25 = = = = = = = = = = +* 289\. Reimplement Cpp Operators +* 283\. Delete thunk +* 281\. Create classes for remaining iiNodes (if appropriate) +* 280\. Decorate CppNodes to use thunk to continue functioning using *either* iinodes or lists +* 282\. Shift code to use the actual iiNodes, removing use of thunk +* 279\. Create thunk that can take both iiNode lists or iiNodes to allow transition to iiNodes +* 278\. Create base class iiNode +* 285\. Document SemVer for Pyxie +* 284\. Create a pyxie.api import, to allow the use of Pyxie as an API. +* 277\. Move all creation of iiNode lists into functions in pyxie.models.iinodes +* 276\. Shift CppNodes into new location +* 275\. Rename existing CppNodes to make things clearer +* 274\. Create pyxie.models.cppnodes +* 0\.1.24 = = = = = = = = = = +* Prep Release 0.1.24 +* 269\. Pynode support for def statements +* 270\. Add a callable type +* 266\. Grammar parsing for function definition succeeds +* 265\. Lexing for function definition succeeds +* 37\. Add special case for function calls like print... #pylang +* 34\. Function call code supports simplified type definitions for externals. #practicalities +* 26\. Block structure of generated C Code is pretty/human friendly/readable #internals +* 3\.5 Analysis code looks for an arduino profile file describing c-types appropriately. +* 3\.6 Arduino profile now configured in separate "profile" interface file +* 215\. v0 no funcs Playful Puppy code compiles, and runs on device correctly +* 214\. v0 no funcs Playful Puppy code generates code +* 213\. v0 no funcs Playful Puppy code analyses +* 264\. Check all Pynodes add all sub nodes as children correctly +* 263\. Bug: "elif" not analysing/generating code correctly regarding context +* 262\. BUGFIX: "else" not analysing/generating code correctly regarding context +* 195\. Functionality of bin/pyxie-dev is in core, not a script +* 0\.1.23 = = = = = = = = = = * 258\. Improve Arduino profile documentation * 260\. Arduino Profile supports HIGH,LOW,INPUT,OUTPUT,A0-A7 * 238\. Update grammar on website/in docs to match current grammar. diff --git a/site/src/panels/trello-cardlist-known-bugs-anti-features.md b/site/src/panels/trello-cardlist-known-bugs-anti-features.md index 43372c0..9b0bb40 100644 --- a/site/src/panels/trello-cardlist-known-bugs-anti-features.md +++ b/site/src/panels/trello-cardlist-known-bugs-anti-features.md @@ -2,7 +2,7 @@ template: mainpanel source_form: markdown name: Trello Cardlist Known Bugs / Anti-features -updated: October 2016 +updated: February 2018 title: Trello Cardlist Known Bugs / Anti-features --- ### Known Bugs / Anti-features diff --git a/site/src/panels/trello-cardlist-paused.md b/site/src/panels/trello-cardlist-paused.md index e271f33..cc3ce31 100644 --- a/site/src/panels/trello-cardlist-paused.md +++ b/site/src/panels/trello-cardlist-paused.md @@ -2,10 +2,8 @@ template: mainpanel source_form: markdown name: Trello Cardlist Paused -updated: October 2016 +updated: February 2018 title: Trello Cardlist Paused --- ### Paused -* 227\. .get_type() should be delegated, not rely on internal pynode details. -* Flesh out a micro:bit profile. diff --git a/site/src/panels/trello-cardlist-pinned.md b/site/src/panels/trello-cardlist-pinned.md index d37dc6a..ed45164 100644 --- a/site/src/panels/trello-cardlist-pinned.md +++ b/site/src/panels/trello-cardlist-pinned.md @@ -2,7 +2,7 @@ template: mainpanel source_form: markdown name: Trello Cardlist Pinned -updated: October 2016 +updated: February 2018 title: Trello Cardlist Pinned --- ### Pinned diff --git a/site/src/panels/trello-cardlist-project-stage.md b/site/src/panels/trello-cardlist-project-stage.md index e9bab30..50d0ef3 100644 --- a/site/src/panels/trello-cardlist-project-stage.md +++ b/site/src/panels/trello-cardlist-project-stage.md @@ -2,15 +2,15 @@ template: mainpanel source_form: markdown name: Trello Cardlist Project Stage -updated: October 2016 +updated: February 2018 title: Trello Cardlist Project Stage --- ### Project Stage * HEADLINE: PRE-ALPHA * DEV STATE: WORKING (USABLE) -* DEV VERSION: 0.1.23 -* RELEASED: 0.0.22 (25 Sep 2016) +* DEV VERSION: 0.1.26 +* RELEASED: 0.1.25 (2 Feb 2018) * LANGUAGE STATE: BARE* -* FOCUS: Practicalities (Arduino Profile real example) - Get the Puppy/Kitten Robot working +* FOCUS: Basic User Functions, Practicalities (Arduino Profile real example) - Get the Puppy/Kitten Robot working * Newsletter created at http://tinyletter.com/sparkslabs diff --git a/site/src/panels/trello-cardlist-rejected.md b/site/src/panels/trello-cardlist-rejected.md index 45f6be2..40c3834 100644 --- a/site/src/panels/trello-cardlist-rejected.md +++ b/site/src/panels/trello-cardlist-rejected.md @@ -2,8 +2,13 @@ template: mainpanel source_form: markdown name: Trello Cardlist Rejected -updated: October 2016 +updated: February 2018 title: Trello Cardlist Rejected --- ### Rejected +* 54\. Review pypi packaging for things we should be doing - Seems OK for now +* 55\. Create pyxie-service for managing batch compilation services - to be folded into Make a Batch Compiler +* 53\. Use https://travis-ci.org/ - No time to do at the moment +* 261\. Consider PEP 484 based type hints - maybe later, but part of point of pyxie is to autodetect types +* 298\. Generated C++ code should a list of fragments and joined at the end. diff --git a/site/src/panels/trello-cardlist-release-backlog-features.md b/site/src/panels/trello-cardlist-release-backlog-features.md index 59e1625..9543f40 100644 --- a/site/src/panels/trello-cardlist-release-backlog-features.md +++ b/site/src/panels/trello-cardlist-release-backlog-features.md @@ -2,9 +2,13 @@ template: mainpanel source_form: markdown name: Trello Cardlist Release Backlog: Features -updated: October 2016 +updated: February 2018 title: Trello Cardlist Release Backlog: Features --- ### Release Backlog: Features -* 0\.1.23 = = = = = = = = = = +* 0\.1.26 = = = = = = = = = = +* 286\. Revise iiNodes to use ECS like ideas +* 288\. Use ECS style iiNodes for type inference and variable collation +* 291\. Rename model.transform to transform.pynodetoiinode +* 32\. C Syntax Tree is a Tree #internals diff --git a/site/src/panels/trello-cardlist-release-backlog-tasks.md b/site/src/panels/trello-cardlist-release-backlog-tasks.md index fef981d..9a38026 100644 --- a/site/src/panels/trello-cardlist-release-backlog-tasks.md +++ b/site/src/panels/trello-cardlist-release-backlog-tasks.md @@ -2,11 +2,12 @@ template: mainpanel source_form: markdown name: Trello Cardlist Release Backlog: Tasks -updated: October 2016 +updated: February 2018 title: Trello Cardlist Release Backlog: Tasks --- ### Release Backlog: Tasks -* 0\.1.23 - - - - - (tasks) +* 0\.1.26 - - - - - (tasks) +* 0\.1.25 - - - - - (tasks) +* 287\. Make repo more visible * 57\. Blog post: Pyxie structure #website -* Release tasks diff --git a/site/src/panels/trello-cardlist-tasks-done.md b/site/src/panels/trello-cardlist-tasks-done.md index 96d8a38..26409c8 100644 --- a/site/src/panels/trello-cardlist-tasks-done.md +++ b/site/src/panels/trello-cardlist-tasks-done.md @@ -2,11 +2,13 @@ template: mainpanel source_form: markdown name: Trello Cardlist Tasks Done -updated: October 2016 +updated: February 2018 title: Trello Cardlist Tasks Done --- ### Tasks Done +* 0\.1.23 - - - - - (tasks) +* 0\.1.23 Release tasks * 0\.0.22 - - - - - (tasks) * 239\. 0.0.22 Release tasks * 0\.0.22 - - - - - (tasks) diff --git a/site/src/panels/trello-cardlist-wip.md b/site/src/panels/trello-cardlist-wip.md index 2f6a0f5..bda99f5 100644 --- a/site/src/panels/trello-cardlist-wip.md +++ b/site/src/panels/trello-cardlist-wip.md @@ -2,8 +2,11 @@ template: mainpanel source_form: markdown name: Trello Cardlist WIP -updated: October 2016 +updated: February 2018 title: Trello Cardlist WIP --- ### WIP +* 297\. Detangle the code transforms out from the cppnodes... +* 18\. Code generation of C literals is muddled up a touch with structural representation #internals +* 293\. Extract the transforms from cppnodes into pyxie.transform.ii2cpp diff --git a/site/src/panels/what-job-does-this-do.md b/site/src/panels/what-job-does-this-do.md index 057e04a..2320729 100644 --- a/site/src/panels/what-job-does-this-do.md +++ b/site/src/panels/what-job-does-this-do.md @@ -7,7 +7,7 @@ title: What Job Does This Do? --- ### What job does / will this do? -The aim of this project is to allow a adults and children to write code in a +The aim of this project is to allow adults and children to write code in a familiar high level language that can then be compiled to run on an arbitrary embedded system - that is devices with very low power CPUs and very little memory. (ie devices too small to host a python interpreter/runtime)