[PTLsim-devel] Multicore problems

Fri Oct 19 14:21:42 EDT 2007

Hi Matt,

thank you very much for your support. I tried the newest version from 
you website, and here are the results.
I've installed (for this test) a fresh Ubuntu Feisty 7.04, on 64-bit 
Intel SMT (2 threads), but with only one core.
After installing all the necessary packages, I had to modify a bit the 
Xen that you have on your website (compile bugfixes), in order to be 
able to compile the Xen on Ubuntu. If you want, I can give you the 
changes, so that you can incorporate them into your Xen version. 
However, they are mostly trivial.


Matt T. Yourst wrote:
> On Thursday 18 October 2007 05:41, Sasa Tomic wrote:
>   
>> Hi,
>>
>> I also have the same problem, even with the unmodified version of PTLsim
>> rev219.
>> I should stress that we are all using multicores (VCPUS>1) in PTLsim. If
>> I use a single-core version, I don't see this problem (this doesn't mean
>> that it doesn't exist). But, for our research, we need multicores.
>> With the rev221 the cores don't even start running if I use more that 1
>> core.
>> If I use only one core, it works correctly.
>
> I'll try to look at this if you give me all the relevant information:
>
> - PTLsim version (you said it was rev 221, but did you modify it?)
>
>   
I downloaded rev. 227 from your SVN. I've only changed the makefile from 
the entire PTLsim source:
stomic at tmserver-desktop:~/new/ptlsim$ svn diff Makefile
Index: Makefile
===================================================================

--- Makefile    (revision 227)
+++ Makefile    (working copy)
@@ -6,6 +6,7 @@
 # Copyright 2000-2007 Matt T. Yourst <yourst at yourst.com>
 #
 
+MACHTYPE=x86_64-pc-linux-gnu
 ifeq ($(findstring x86_64,$(MACHTYPE)),x86_64)
 __x86_64__=1
 endif
@@ -20,7 +21,7 @@
 # but can still run 32-bit code and guest operating
 # systems. See the manual and FAQ for details.
 #
-# PTLSIM_HYPERVISOR=1
+PTLSIM_HYPERVISOR=1
 
 CC = g++
===================================================================
I had to do the change, because the Makefile didn't recognize that this 
is a 64-bit machine. I don't know if this is machine specific or Ubuntu 
specific.

> - Host kernel version (are you using the 2.6.22.6 kernel we provided?)
>
>   
I'm using the kernel from your website, with your config file and I've 
just included some drivers that I needed for booting this specific 
machine (some SCSI and SATA drivers), but this is irrelevant to the Xen, 
at least I think so.
> - Guest kernel version (are you using the 2.6.22.6 kernel we provided?) and 
> disk image (if you're using something other than the 2.6.22.6 kernel inside 
> the disk image we provided as an example).
>
>   
the same as host kernel
> - Xen hypervisor and tools version (please give the date of the archive we 
> provided on the PTLsim web site)
>
>   
The version downloaded from the website:
wget https://ptlsim.org/xen-3.1-ptlsim.tar.bz2

> - Type of host machine. In our lab we have dual-core Athlon 64 X2's, Core 2 
> Duos and Core 2 Quads to test it on. This is important in case there's a race 
> condition, since that may show up on some CPUs and not others because Linux, 
> Xen and PTLsim all use different code sequences on different CPUs.
>
>   
We had the same symptoms on Intel Xeon 4x2 cores (that's probably the 
Core 2 Quad, I'm not sure), with SuSE 10.1 installed
and on this Fujitsu - Intel 1 core SMT (2 threads) with Ubuntu 7.04 
installed. So, I think that it's not distribution or host machine related.

> If you can download and try the latest versions of PTLsim, kernel, sample disk 
> image, Xen hypervisor and tools (currently on our web site), that would be 
> very helpful. If you use an older version of PTLsim, I'll have to reconfigure 
> our machines to run an older version of the hypervisor, which is a real pain.
>
>   
Now, the most important thing is the Xen config file (for running the 
virtual machine).
-------
# -*- mode: python; -*-

name = "s"
memory = 256
extra = "norandmaps nousb noide xencons=ttyS console=ttyS0 init=/sbin/init"

kernel = "/project/ptlsim-checkpoints/2.6.22.6-mtyrel-64bit-xen/linuxkernel"
ramdisk = "/home/stomic/new/img/sample-vm.img"
builder='linux'
vcpus = 2              # or more!
-------
If I put "vcpus = 1" everything works perfectly. If I put 2 or more (I 
tried up to 4 today, and up to 16 in the past), the results are 
unpredictable.
I figured out that in some cases the simulator works with 2 or more 
processors. In that one case when it did work, the kernel was running on 
CPU 1 (not 0 as usually).
But, almost in every case, the kernel (interrupts and stuff) works on 
CPU0, and then the simulator is stuck - i.e. it doesn't simulate anything.
This is completely separate bug from the one when commits: and 
insns/sec: are 0 all the time. The CPUs wake up for timer interrupts and 
then go back to idle.
Example:
Completed      99304151 cycles,        134687 commits:   6922477 
Hz,         0 insns/sec: rip (blocked) 
(running)                                             Completed     
100735050 cycles,        134687 commits:   7154489 Hz,         0 
insns/sec: rip (blocked) 
(running)                                             Completed     
102171853 cycles,        134687 commits:   7184011 Hz,         0 
insns/sec: rip (blocked) 
(running)                                             Completed     
103600747 cycles,        134687 commits:   7144464 Hz,         0 
insns/sec: rip (blocked) 
(running)                                             Completed     
105090626 cycles,        134687 commits:   7449391 Hz,         0 
insns/sec: rip (blocked) 
(running)                                             Completed     
106530070 cycles,        134687 commits:   7197212 Hz,         0 
insns/sec: rip (blocked) 
(running)                                             Completed     
108664014 cycles,        138552 commits:  10669714 Hz,     19324 
insns/sec: rip (blocked) 
(running)                                             Completed     
110177670 cycles,        138552 commits:   7568277 Hz,         0 
insns/sec: rip (blocked) (running)

I hope this is enough details to get you started. If you need anything 
else, please ask me, and I'll try to give some more information.


thank you,
Sasa

-- 
Saša Tomić
BSC - Barcelona Supercomputing Center
c\ Jordi Girona 29, Nexus I, 08034 Barcelona, España
Tel.: ,  
http://www.bsc.es

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://ptlsim.org/pipermail/ptlsim-devel/attachments/20071019/12b8d76c/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
Url : https://ptlsim.org/pipermail/ptlsim-devel/attachments/20071019/12b8d76c/attachment.bin 
