minetestmapper-python

Use python to make minetest maps from worlds.

License

• sectors2sqlite.py: see lgpl-2.1.txt
• other code: see LICENSE

Authors

• The github.com/minetest team: minetestmapper.py and sectors2sqlite.py (see CHANGELOG.md)
• spillz: the original minetestmapper-numpy.py only
• Jake Gustafson

Goals:

• Conform syntax and features to the new official https://github.com/minetest/minetestmapper which is in C++
• Add all features (especially standard output of geometry) from minetestmapper-numpy.py to minetestmapper.py
• Maintain minetestmapper-numpy.py (minetestmapper.py is considered low priority but seems to work better for sqlite worlds)
• Keep pace with changes to Python and minetest world format

Reason for forking

• minetestmapper.py: The official python mapper util was discontinued by the minetest team because making maps using python is difficult to do efficiently considering the amount of data, and it was no longer considered necessary since they are now maintaining a C++ port at https://github.com/minetest/minetestmapper (the problem was eliminated by eliminating the python mappers from the repo, though spillz had significantly improved the performance using numpy in a fork).
• minetestmapper-numpy.py: Unfortunately, support requests directed to spillz have been on his official thread for a long time now, so I am placing that here. In addition, there is no reason to fork minetest to work on these mappers (spillz' version is in his fork of minetest), as the python mapper is no longer there. I suggest to spillz that if he continues his work on it, that he forks from here for that reason.
• minetestmapper: As for the C++ version of minetestmapper, for now it lacks some of the functionality of these mappers, and must be compiled from source in order to be installed. In addition, many steps are required in order to create a colors.txt file that covers all of your mods. Many authors have created some great colors.txt files, so all of that will be combined into this project's colors.txt (for reasons such as VenessaE's colors.txt makes prettier maps). This project can continue alongside the C++ version but hopefully the C++ version will integrate some important features so it can be used by web map projects.

Replaces the following deprecated projects:

• minetestmapper.py (and related sectors2sqlite.py) formerly located at https://github.com/minetest/minetest/tree/master/util
• minetestmapper-numpy.py stranded as a pull request (in spillz' minetest fork) then apparently abandoned
  • See the official topic: numpy-izing minetestmapper which links to: https://github.com/spillz/minetest/raw/master/util/minetestmapper-numpy.py

Differences from upstream (deprecated) versions

• Exceptions are more clear.
• It runs under Python 3.

Requirements

• Python Imaging Library: http://www.pythonware.com/products/pil/
• The colors.txt should be generated by https://github.com/poikilos/EnlivenMinetest/blob/master/mtanalyze/minetestinfo.py for the most complete colors list (unless you use the more complicated minetestmapper method) and prettiest map until that code is moved from EnlivenMinetest to the minetestmapper-python project.

Developer notes:

(Poikilos)

minetestmapper-numpy.py

• self.db is an iterator from get_db (which returns either an SQLDB or LVLDB object).
• The db iterators stop (via break) whenever not r[0], but still there are problems (below is from __iter__ in LVLDB).

        for k in self.conn.RangeIter():
            #if k is not None and len(k)>0:
            try:
                # print("getting int from first index of value "+str(k))
                # SOMETIMES has an integer, such as getting int from first index of value ('905945065', '\x19\x01\x02\x02x\x9c\xedX\xdbu\xdb0\x0c\xa5\xa2Jv\xbf|\xbaBW\xe8\n]!+t\x85\xac\x90\x15\xbc\x82V\xd0\n\x1a\xaa\xa4\x14\x8a\xc0%\xf8\x90DK~\xe4\xf2\x9c\xc4 \xf8\x00pA\x90\xb6R\x04\x95noc\xab\xe6\xcf\x1cV_\xa2\xd5\xba\xfd\x88\xe8\xa6\xe6\xfa*\xd2$\xbd?\x17\xf5\xae\xb7!\x9f[-\xb5_\xad\x9e\xff\xd7D\xa2\xcd\xaeE\xc7\xd7L\xc3wkE\x1bC>\xe6\xeb\x9d\xce\x1f\xc5\xfd\xaf\xa3-\x86*\xa8w;\xb9]\x97\xeb\xeb\x05\xfaJ\xc8\xc3p~H\xeb\xf3H4\xe4SK\xf8\xb7\xac\xc9\xec[\x96\x1b\xc8\x1a\x9f\x9fv^\xc7\xcf\x0cjK\x93`\xffM\\\x9d\xe7\x06\x8e\x90\xf2\x7f\r\xffa\xfdr\xfen\xab\xaf\xbc\xfc\xa8\xbc\xf9~\x1cm\x0cN\x1e\xdf\x86\x97Fau\xa0\x8d\xf7\x86\xf9i2\xf8[\xaa\x0f\xd5\xb7\xd2\xfc\xc7Aw<V_\x89\xfa\xbc\xbb\xca\x9dxW\xf3\xdd\xc9\xe5\x15>\xdc\xe8Z|~\x13\xcd\x8e4\x7f\x92\x17!\xfdY\xe0W\x9e5\xe5eS\x88\xffc\xf52\xfb\xf45\x93\xc3?2\xee\x98Oq\xefF\xd7\xde|~\xfe\xa5\xf3G\xad<\x0bc\xb8^:\xbfUt\xfd0\xff\xcd\xc6\xf3o9\xb8\xad>\xc4n\x9e>\x9d\x01\x9c\xfd:\xc0i\x0e\xff\xf4\xe4\xbb\xdc\xe1\xaf\x91\x18\xff\x12\xbb\xf5\x1c\xa5\xd0|\xe7\xa5\xac\x0f\xf1O\xe7\xafG\x9c\xbd=\xf4)\xfeC5\x96GE\xee\xcdc\x9e\xe6JM\xfeO\xf3+\xf2\x16\t\xf1\x1f\xe3o\xf22\xc5\x7fx\x87X\xed\xd8\xce\xff\xf1\x88\xf3\xaf\xb2\xf8\x975\xfe[ 7\x03h\x1eL6\xdao\xacM \xfe\x95\xa8\xa3\xfc\x84\xf2\'\xa5\xb7\xde\xc8\xab?>\xff)\xa4_\x00\xa1\xb6\xf4\xf4\xe3\xf9\x9fz-&\xf6OQ\xfe$\x8e\xa8\xbeN\xcc\x8f\xf1\xefn\xfco\xfei\xe3\xbf\x18\xf8\x19\xe0\xeb9\xf3\x16\xee\xd4\xf3L@\xfd\xfa\xf3\xdd\x88\x0c\xa6\xe6\xc7o\xffo\xfe\xf17#\xcc\x00I\xdf\xea\xdb\xd4\xf2\x8c;\xe1-\xe0\x90\x8a\x7f\x9a\x9f5z\xbc\xeb^\x8d\xff\x18\xbb\x9c\x7fiL\xcb\xd6\xc1\x1a\x80\xf1\x93bLQ\x82_\xac\x00\xd2|:\x86Z-\xe9\x9f\x1dq~9\xff\xfe\x88\xd6[\x0bO9\xee\x869\x80H\x9d\xbfZ\xfd\xdc8\x9f\xf3\x8b\x1e\xe1;\xe0\xd9\x11\xe7\xf7\r\xc6\xe6\xf3_\x0b\xe7\xbf\x0c\xffe\xf5\xe8\x13V\x90gG\x95\xc8\x00\x1c\xcb\xb5\r\xac\xe6\xa28\xfd\xb6\xd7\x82>\xc5\x7f\xbc\xbe\xdfB_\x01\xfb\xa9\xfcy6 \xff\xc80"v\xfe\x9d\xbe\r\xc6\xef\xde\xce\xbf_\xff\xc5\xf9\xc30\x98\xbf=\x915\xfanp\xe8\xfb\xbe\xbb\xf6\x06V\xa9\xc7u\x1a_\xf28\xefz\xd5"1\xe7S\xcb\xfd\xacU\xeaCwt\xbdZ\x86?\xbf\x7f9\xe1\xe3\xdf\xfb_\xda\xf1\t\x1d\\66*"_\xf3w\x1d\xadv\x9e)\x13\x005\xbao\xe5n\xf4\x9f\xc9\xd4}\x13M#[\xbd\x91\xa7x\xe4\x1b\xa1\x11\xf7\xb7\xbb\xb2\x8e^\x81\xdc\xd9\tv\xf857\x04\xa3\xe3\xd4\x7fc\xff\xc8>\x91\xbb.!Sg\x07\x1b\x0f\x9bQYv\x00\xbf\xe0_?\x07`\xdab\x96\x95(k1?\x05H^\x7f-\xd63\xf2\xf4\xe2\x1d\xc8\xa8\xe7\\\x8f\x1d\xf4Hu\x19Vh\xf3\xa9\x08\xfe\xcd\x0e\xba-|\x99Y@\xc58\xc0}\xb3Y\x9f\x90q<\xa4\xfa\xc0\xdc\xcf\xcd\x00\x06t\xc0x\xa4\xdeAF=5\xa9[T\x03\x16\xca8\xdf_\x90#\xdf\x94y\x05H\x9a\x94\xacX\x90s\x93\xce\x0e/,+\x8c\x80\x1aK\xca\x88\xd56\xf1\x99~R\xf1\x94\\\x13\xf3\x92@\xff{\x87\x9dv\xdfa\x9b\xe8\xfe\x88W\xf6\x7f\xec\x83\x00\x98ke\xc1\x19]\xbc\xff\x8d\x96\xce\xde\xdf\xab\x7f~\x00z\x16\x80\xa1\\j\xac\xad\xba%\xe1\xd7\xff\x01N\x80)\xd2,\x00\xe5,\xbe\x03\xffi\x04l\x07V\x00\xffV+\xbbs\x99\xd56\x9b\xb1\xbb\xff\xea>\xfc\xf7x\x18\xb8\xff\x90\xff\xc5\xfd/\xb4\xd8f;B\xfe\xef\x9c\xff\x97\xcb\xf8\xe7\xe2:6mr\x19\xb1i\x89cP\xc6\xe6\x0b.v\x1aQd\xed\x07\x84\xcd\x84\x87\xcc\x88\xed\xb8\x88\x1fo\x85sx\xf7c@\r\xd8!\x03\xbc-v\x0e@b\xfb\xe2\xd6\xa4Czl\x06@\xd5\xdbl\x0c\x96\xd1\xf4=sl\xd9)_\xf5\xbd\x8b\xe4\xf0#\xfe\x8d\xf58kP\xf9A\x1fN\xabq\x86\x00<\xea\xcbq-\x8c\xbb\xe73\xf6\x1dc\xcb\x11\x98\xbe*x\xaf\x96\xbb{\xc6\x94\x82\xfcU\xc0{5\xbc\x9a\xff\xc9K\xfcF\xe6\xec\x8e\xd3\xe9\xbe\xbe\x0c\xfe\x07\x80pRrx\x9cc\x00\x00\x00\x01\x00\x01\x00\x00\x00\xff\xff\xff\xff\x00\x00\n\x00\x00\x00\x13default:lava_source\x00\x01\x00\x14default:lava_flowing\x00\x02\x00\x03air\x00\x03\x00\rdefault:stone\x00\x04\x00\x17default:stone_with_mese\x00\x05\x00\x17default:stone_with_coal\x00\x06\x00\x0etechnic:marble\x00\x07\x00\x14moreores:mineral_tin\x00\x08\x00\x17default:stone_with_iron\x00\t\x00\x17moreores:mineral_silver\n\x00\x00')
                val = k[0]
                # if k[0][:2]=="\\x":
                    # in leveldb, minetest stores \x before every byte of the value, so remove all and prepend 0x so python can convert to int
                    # but this doesn't work (resulting val still throws exception)
                    # val = "\\0x" + k[0].replace("\\x", "")
                x, y, z = getIntegerAsBlock(int(val))
                yield x, y, z, val
            except Exception as e:
                print("Could not finish getting int from first index of value "+str(k))
                # SOMETIMES prints tons of output such as Could not finish getting int from first index of value ('\x00\x00\x00\x0e\x00\x02\x85 ', '\x19\x08\x02\x02x\x9c\xed\xc11\x01\x00\x00\x00\xc2\xa0\xf5Om\x0c\x1f\xa0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\xb7\x01@\x00\x00\x01x\x9cc\x00\x00\x00\x01\x00\x01\x00\x00\x00\xff\xff\xff\xff\x00\x00\x01\x00\x00\x00\x06ignore\n\x00\x00')
                raise e