Second Day, Fibonacci Numbers running

Yesterday was a day of excessive coding. After finishing the W_Object model together on the projector, we worked in four teams on three parallel tracks:

• loading a real Squeak image (Adrian K, Carl Friedrich)
• interpreter loop and bytecode dispatch (Armin, Adrian L, Lukas, Toon)
• implementing numbered primitives (arithmetic) (Niko, Tudor, Oscar)

Image Loading

There are certainly worse save file formats than Smalltalk images, however to conserve bits must have been very fashionable back then in the 80s. There are three formats for object headers, the compact one being the most crazy:

(2 bits) header type
(6 bits) word size
(4 bits) object format
(5 bits) compact class id
(12 bits) identity hash
(3 bits) used by gc

with that, memory footprint of empty instances can be as low as 4 bytes only, but at the expense of having 12 bits for identity hashes only! We are thinking about integrating a transition from 12 bit to full word hashes in the VM. Legacy instances from the Squeak image file will retain their 12 bits (to not corrupt existing hash tables) but any newly allocated object will be initialized with full 32 bit hashes. The limitation being that we lose backwards compatibility to the Smalltalk-80 save format.

At the end of the day, we were able to load in the raw data of all objects in the image, and to verify that all header information and pointers are correctly set. Today the image loader will be finished by instantiating one W_Object (or subclasses) for each of these image chunks.

Interpreter and Bytecodes

Armin, Adrian L, Lukas, Toon started implementing the bytecode dispatch loop and the context frame class. Once they got this they split in two groups and started implementing the bytecodes (most of which are there by now, apart from some which call primitives). At one point during the afternoon they managed to copy Squeak's bytecode for the Fibonacci function and run it.

Primitives and Base Image

The trio of Niko, Oscar and Tudor began implementing the base image, including class objects. They implemented templates for the math operations. This part is more or less complete, except for promoting from small to large integers.

However there are still plenty of primitives to go for the next days.

Some sprint pictures

Here are some pictures of the sprint today (of course all sprint pictures are really pretty similar, it's people in rooms around laptops).



Above, people are (clockwise, starting lower left corner): Niko Matsakis, Armin Rigo, Toon Verwaest, Tudor Girba, Oscar Nierstrasz, Adrian Lienhard.



Above, Armin Rigo, Lukas Renggli, Toon Verwaest, Adrian Kuhn, Carl Friedrich Bolz (absent, taking the photo), Adrian Lienhard, Niko Matsakis, Oscar Nierstrasz.



This important picture is showing the attempt of teaching the Swiss German speakers to pronounce "interpreter" correctly.

How to check out and try PySqueak

To try it out, you need subversion and Python version 2.4 or 2.5. You first have to check out PyPy:

    svn co http://codespeak.net/svn/pypy/dist pypy-dist

Warning, this checkout is relatively big. Another way if you only want to look at the sources is via https://codespeak.net/viewvc/pypy/dist/pypy/lang/smalltalk/.

The Smalltalk code is in the pypy/lang/smalltalk subdirectory; you can run its tests (72 so far) as follows:

    cd pypy/lang/smalltalk
    ../../../py/bin/py.test

Happy poking around :-)

Codespeak is back up again

Codespeak's repository is back up again and we imported the work that we did today into the pypy/lang/smalltalk directory.

Anonymous' questions answered

The questions that an anonymous commenter asked on yesterdays post are worthwhile to answer, so I decided to put them into a post of its own.

If someone could explain the differences between the three goal options in terms of what they yield, that would assist my understanding of PyPy no end!


To implement a Squeak-bytecode interpreter in RPython


This yields an interpreter, on PyPy's back ends, that executes user level Squeak code (compiled in Squeak) -yes?

Yes. The interpreter will be runnable after translating them with PyPy's backends, but also be executable on top of CPython, of course (which is slow, but good for testing). It seems we will be able to load (at least rudimentary) Squeak images.


The Squeak bytecode would be PyPy jitable -yes?

Very likely, yes.


Question: Would this interpreter necessarily need to implemented the interpreter-objectspace separation (plus flowspace)? Or is this separation only required if we want to translate (including jit) Squeak bytecode?

No, we are not implementing this separation in the Smalltalk interpreter. The separation is specific to PyPy's Python interpreter because we want to use it to analyze RPython. The Smalltalk interpreter (as most other custom interpreters) will be jittable without this separation.



* to define and implement an RSqueak as PyPy frontend

This allows translation of RSqueak code to PyPy backends -much as slang does to C (but allowing more dynamism than slang). Yes?


Absolutely. Of course it really depends on lots of factors, so who knows, without trying :-). The RSqueak would have been to be really object-oriented as opposed to disguised C (like slang).



One might then write an interpreter for Squeak in RSqueak -and the result would, ceteris paribus, be identical to 1 above in terms of capabilities. Yes?

Probably, yes. Of course for some Smalltalkers it will probably seem preferable, since they don't have to deal with this icky Python language too much :-).

This second option could also be used as a starting point for a completely Smalltalk-based toolchain.


* to write a Squeak backend for PyPy

So we would have a Python interpreter running on the Squeak VM? -executing Python bytecode on top of Squeak (which would be pretty slow) -or is there a way of getting from Python source to *Squeak* bytecode a la Jython/IronPython on JVM/CLR?

The former. And yes, the main reason for not doing this is because it would be too slow.

Codespeak repository down

The codespeak server is currently down, so it's not possible to look at the code at the moment (and also not at the images and talks linked to in the last post). The sprinters have switched to a non-public repository, we will post something here if the situation changes.

First day, Discussions

Bern Sprint, First Day, October 22

So, it seems customary for the Squeak community to use such incredibly advanced technology as RSS feeds and blogs to disseminate their sprint reports, which means us old-fashioned PyPyers just have to follow suit.

Today started the first ever Pypy-Squeak-collaboration sprint, kindly hosted in Bern by the Software Composition Group. The sprint aims to bring people from two different communities together (not without the unavoidable clash of culture, the first wars on zero- versus one-based indexing were already fought today) to learn about each others projects and to explore collaboration possibilies.

Morning was spent setting up the sprint room and general computer infrastructure like Squeak's VMMaker and Pypy. It seems that bootstrapping Squeak is certainly faster then bootstrapping Pypy (and uses way less memory) but not without certain problems. Adrian Kuhn tried to distribute an USB stuck containing a zip archive created under Windows. This turned out to be a bad idea, as obviously the notion of execution access permission is not known to Windows but crucial for Unices. In the end we finally succeeded. The PyPy checkouts of the Squeak guys on the other hand went quite smooth. Nobody tried to translate PyPy yet, though.

After lunch, Armin and Carl Friedrich gave various talks and demo for the SCG crowd (including some increasingly confused students). The first talk covered sprint-driven development in a two-slide presentation, including visualizations of software evolution, then we delved into the technical topics by re-giving our Dyla talk and demos.

After a break, we started brainstorming. We identified the following options as possible main goals for this week's sprint

* to implement a Squeak-bytecode interpreter in RPython

* to define and implement an RSqueak as PyPy frontend

* to write a Squeak backend for PyPy

even though option 2 was favoured as long term goal, we decided to go for the first option. To complete the second option, we felt, that we miss yet an understanding of the second option's trade-offs. Now, we do option one to gain insight into and a feel for the second option. We broke the first option down into smaller steps, as shown on this picture

Next Carl Friedrich quickly introduced the Squeak crowd to RPython, whereas Adrian Kuhn gave a quick and dirty introduction into the Smalltalk object model: "objects all the way down".

Finally, everyone gathered around Armin's laptop, who started hacking the very beginning of said object model into a set of RPython classes. Always writing tests first, true to Test Driven Development (while the rest of the crowed watched in amazement how the svn comments are announced live on PyPy's IRC channel). If you are interested in this very first prototype, please refer to the svn repo.

Please stay tuned for more news tomorrow.

Carl Friedrich Bolz and Adrian Kuhn