emacs and large-address awareness under recent snapshots

[Ken Brown]

On 8/9/2011 11:21 AM, Corinna Vinschen wrote:
> On Aug  9 10:23, Ken Brown wrote:
>> On 8/9/2011 10:12 AM, Ken Brown wrote:
>>> On 8/9/2011 7:19 AM, Ken Brown wrote:
>>>> (gdb) thread 1
>>>> [Switching to thread 1 (Thread 19828.0x447c)]
>>>> #0  0x00622ee0 in morecore_nolock (size=1052672) at gmalloc.c:703
>>>> 703           while ((__malloc_size_t) BLOCK ((char *) result + size)>
>>>> newsize);
>>>> (gdb) p /x size
>>>> $1 = 0x101000
>>>> (gdb) p /x heapsize
>>>> $2 = 0x80000
>>>> (gdb) p result
>>>> $3 = (void *) 0x807d0000
>>>> (gdb) p newsize
>>>> $4 = 0
>>>> (gdb) p _heapbase
>>>> $5 = 0x816000 "\202"
>>>> (gdb) p _heapinfo
>>>> $6 = (malloc_info *) 0x80060000
>>>>
>>>> Is _heapbase the problem?  This is initialized to _heapinfo at the first
>>>> call of malloc and is never changed.  _heapinfo presumably points into
>>>> the static heap at that point.  (_heapinfo is later changed as a result
>>>> of realloc.)  This low value of _heapbase is used in the BLOCK macro.
>>>
>>> Here's what I think is happening.  When temacs.exe is running during the
>>> build process (see my explanation of this earlier in the thread),
>>> malloc_init is called and _heapbase is set.  At this point, temacs is
>>> using its own static buffer as the heap, and _heapbase gets the value
>>> 0x816000.  This gets dumped as initialized data into emacs.exe, as does
>>> the value __malloc_initialized = 1.  Now when emacs.exe is run, it sees
>>> that malloc has already been initialized, so _heapbase retains its
>>> value, which is no longer appropriate.  All code relying on the BLOCK
>>> macro is now invalid.
>>>
>>> AFAICS, this has always been wrong.  But the error didn't have dramatic
>>> consequences until the heap was put into high memory.
>>>
>>> I'm not sure what's the best way to fix this (assuming my analysis is
>>> right).  Would it be enough to set __malloc_initialized to 0 before
>>> dumping?  That would force emacs to reinitialize and get the correct
>>> value of _heapbase.
>>
>> No, that's too simple-minded.  I just tried it, and emacs aborted.
>> This seems like a mess.
>
> What happens if you remove the Cygwin-specific call to bss_sbrk in
> __default_morecore?

That will mess up dumping.  The point of using bss_sbrk and simulating 
the heap in a static buffer is that whatever has been stored in that 
buffer gets dumped into emacs.exe as initialized data.  See unexcw.c.

 > In theory that should also break, as long as
> temacs isn't also build large address aware.  The only difference,
> _heapbase = 0x20000000.  But if temacs gets build with large address
> awareness set, _heapbase should become 0x80000000.

That would make _heapbase (which is part of the dumped emacs.exe) depend 
on the build system.  Obviously, as you say below, _heapbase needs to be 
determined at run time.

> However, whatever you do, it will not really work.  Keep in mind that
> the large address awareness only makes sense (and has any effect!) on
> systems which provide a large address area.
>
> To me the bottom line here is, that emacs is doing the wrong thing.
> There are a couple of assumptions how a system maintains memory, which
> are just not valid on all systems.  The malloc initialization and the
> assignment of the heapbase (the first call to sbrk(0)) should happen
> in emacs every time it starts.

That makes sense to me.  I thought that was what I was accomplishing 
(for Cygwin) by setting __malloc_initialized to 0 before dumping.  I'm 
not sure why it didn't work.  In any case, the fix shouldn't be Cygwin 
specific.  It's probably time to report this as an emacs bug.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple