cloudflare
/
luajit-mm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20 Commits
1 Branch
0 Tags
164 KiB
C 75.6%
C++ 20.8%
Makefile 3.6%
 
 
 
Go to file
Shuxin Yang f6c4ca0d94 This change introduces "block cache" to avoid the cost of TLB manipulation, 
and page initialization via zero-filling.

  Suppose a allocated block B1, whose virtual address is [ad1, ad2], is going
to deallocated. One Linux, it seems the only way to deallocate the pages
associated with the block is to call madvise(..MADV_DONTNEED...) (
hereinafter, call it madvise() for short unless otherwise noted).

 madvise() *immediately* remove all the pages involved, and invalidate the
related TLB entries. So, if later on we allocate a block overlapping with
B1 in virtual address; accessing to the overlapping space will result in
re-establishing TLB entries, and zero-fill-pages, which is bit expensive.

 This cost can be reduced by keeping few blocks in memory, and re-use the
memory resident pages over and over again. This is the rationale behind the
"block cache". The "cache" here may be a misnomer; it dosen't cache any data,
it just provide a way to keep small sum of idle pages in memory to avoid
cost of TLB manipulation and page initialization via zero-filling.
 		2014-08-26 22:24:30 -07:00
obj rudimentary implementation luajit mem-managment for taking full advantage of lower 2G memory 2014-08-07 11:05:29 -07:00
LICENSE Initial commit 2014-08-07 10:48:40 -07:00
Makefile This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
README.md rudimentary implementation luajit mem-managment for taking full advantage of lower 2G memory 2014-08-07 11:05:29 -07:00
adaptor.c This trivial change is just to mark the revision which suprisely 2014-08-25 11:15:41 -07:00
block_cache.c This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
block_cache.h This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
chunk.c This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
chunk.h This change include two parts: 2014-08-22 11:28:53 -07:00
demo.c rudimentary implementation luajit mem-managment for taking full advantage of lower 2G memory 2014-08-07 11:05:29 -07:00
lj_mm.h This change include two parts: 2014-08-22 11:28:53 -07:00
ljmm_conf.h Miscellaneous minor tweaks 2014-08-24 10:42:35 -07:00
mem_map.c This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
page_alloc.c This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
page_alloc.h This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
rb_test.cxx This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
rbtree.c This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
rbtree.h This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00
unit_test.cxx This change include two parts: 2014-08-22 11:28:53 -07:00
util.h This change introduces "block cache" to avoid the cost of TLB manipulation, 2014-08-26 22:24:30 -07:00

README.md

luajit-mm

Luajit take full advantage of lower 2G memory on AMD64 platform.

Rumdimentary implementation. Not yet fully tested. Not yet cleanup the code. Quite a few optimizatio is not yet implemented.

Immediate todo

o.Refine and finish this README.
o.test, add enhancements.

problem statement:

On Linux/x86-64 platform, Luajit can use no more than 1G memory due to the combination of bunch of nasty issues. 1G is way too small for server-side application.

This package is trying to replace mmap/munmap/mremap with hence provide up to about 2G space.

Basic ideas

o. When a application, which contain luajit, is launched, reserve the the space
   from where `sbrk(0)` indidate all the way to 2G.

o. Perform page allocation on the reserved space. the mmap/munmap/mremap is built
   on this page allocation. Currently, we use buddy allocation for page allocation
   with some optimizations in an attemp to reduce working set.