nanospin_*() functions and SMP

bridged with qnx.rtos
Post Reply
Davide Ancri

nanospin_*() functions and SMP

Post by Davide Ancri » Fri Jul 20, 2007 10:22 am

Hi all

are the nanospin() functions safe in a SMP environment? I suppose they
spin loop reading the clocks counter of the microprocessor, but what
happens if the thread gets preempted and then moved to another core?

The value returned by rdtsc instruction is not coherent across different
cores, but nanospin() cannot disable interrupts to avoid preemption (I/O
privileges are not mandatory).

Davide

--
/* Ancri Davide - <davidea at prisma-eng dot it> */

Mario Charest

Re: nanospin_*() functions and SMP

Post by Mario Charest » Fri Jul 20, 2007 1:05 pm

"Davide Ancri" <falsemail@nospam.xx> wrote in message
news:f7q1go$mdn$1@inn.qnx.com...
Hi all

are the nanospin() functions safe in a SMP environment? I suppose they
spin loop reading the clocks counter of the microprocessor, but what
happens if the thread gets preempted and then moved to another core?

The value returned by rdtsc instruction is not coherent across different
cores, but nanospin() cannot disable interrupts to avoid preemption (I/O
privileges are not mandatory).
If QSS did their job right nanospin should prevent thread migration up entry
and restore the thread migration setup on exit? Maybe you can check for
this with the system profiler.
Davide

--
/* Ancri Davide - <davidea at prisma-eng dot it> */

Colin Burgess

Re: nanospin_*() functions and SMP

Post by Colin Burgess » Fri Jul 20, 2007 1:47 pm

nano_spin doesn't disable interrupts, so you may be prempted - but that has nothing to do with SMP.

If you need a sufficiently accurate busy loop to handle hardware then you certainly will need to
disable interrupts around that piece of code, whether SMP or not.

Likewise, you should really call nanospin_calibrate(1) (ie with interrupts disabled) if you really
need super accurate timing.

Mario Charest wrote:
"Davide Ancri" <falsemail@nospam.xx> wrote in message
news:f7q1go$mdn$1@inn.qnx.com...
Hi all

are the nanospin() functions safe in a SMP environment? I suppose they
spin loop reading the clocks counter of the microprocessor, but what
happens if the thread gets preempted and then moved to another core?

The value returned by rdtsc instruction is not coherent across different
cores, but nanospin() cannot disable interrupts to avoid preemption (I/O
privileges are not mandatory).

If QSS did their job right nanospin should prevent thread migration up entry
and restore the thread migration setup on exit? Maybe you can check for
this with the system profiler.

Davide

--
/* Ancri Davide - <davidea at prisma-eng dot it> */


--
cburgess@qnx.com

Davide Ancri

Re: nanospin_*() functions and SMP

Post by Davide Ancri » Fri Jul 20, 2007 2:43 pm

Mario Charest wrote:
If QSS did their job right nanospin should prevent thread migration up entry
and restore the thread migration setup on exit? Maybe you can check for
this with the system profiler.
Are there specific system calls (really fast, in possible!) those can be
used by any thread to ask for this service?

I'm trying to port some rdtsc-based utilities from single to multi-core...

Davide

--
/* Ancri Davide - <davidea at prisma-eng dot it> */

Davide Ancri

Re: nanospin_*() functions and SMP

Post by Davide Ancri » Fri Jul 20, 2007 3:36 pm

Colin Burgess wrote:
nano_spin doesn't disable interrupts, so you may be prempted - but that
has nothing to do with SMP.

If you need a sufficiently accurate busy loop to handle hardware then
you certainly will need to
disable interrupts around that piece of code, whether SMP or not.

Likewise, you should really call nanospin_calibrate(1) (ie with
interrupts disabled) if you really
need super accurate timing.
Colin, this time I'm not playing with hardware, I only want some
microseconds busy loop.

If nanospin() can be preempted and my CPU affinity mask allow my thread
to be migrated to another core, how can the interrupted nanospin()
restart its spin loop being sure of the temporal coherency of the rdtsc
value? rdtsc (or neutrino ClockCycles()) returns different values if
executed contemporarily on different cores.

If there's a way to easily protect pieces of code against thread
migration without blocking priority preemption I would like to know it... ;)

Else I would like to know how nanospin() guarantee correct timings.

Davide

--
/* Ancri Davide - <davidea at prisma-eng dot it> */

Colin Burgess

Re: nanospin_*() functions and SMP

Post by Colin Burgess » Fri Jul 20, 2007 4:19 pm

Davide Ancri wrote:
Colin Burgess wrote:
nano_spin doesn't disable interrupts, so you may be prempted - but
that has nothing to do with SMP.

If you need a sufficiently accurate busy loop to handle hardware then
you certainly will need to
disable interrupts around that piece of code, whether SMP or not.

Likewise, you should really call nanospin_calibrate(1) (ie with
interrupts disabled) if you really
need super accurate timing.

Colin, this time I'm not playing with hardware, I only want some
microseconds busy loop.

If nanospin() can be preempted and my CPU affinity mask allow my thread
to be migrated to another core, how can the interrupted nanospin()
restart its spin loop being sure of the temporal coherency of the rdtsc
value? rdtsc (or neutrino ClockCycles()) returns different values if
executed contemporarily on different cores.
nanospin() doesn't use rdtsc - only nanospin_calibrate() does that. Also, nanospin_calibrate()
does lock itself to one CPU while calibrating.

nanospin() essentially is a for(;;) loop.
If there's a way to easily protect pieces of code against thread
migration without blocking priority preemption I would like to know
it... ;)
How can you ever guarantee accurate timings then if you don't disable preemption?
Else I would like to know how nanospin() guarantee correct timings.
If you are willing to accept preemption then migration to another CPU should not make it any worse,
since it's still just a for loop

--
cburgess@qnx.com

Mario Charest

Re: nanospin_*() functions and SMP

Post by Mario Charest » Fri Jul 20, 2007 5:44 pm

nanospin() doesn't use rdtsc -
Ah!!! For some reason I assumed it was.

Evan Hillas

Re: nanospin_*() functions and SMP

Post by Evan Hillas » Sat Jul 21, 2007 3:22 pm

I'd say disable interrupts. It's the only way to be sure. ;) The other way is to setup a separate hardware timer attached to an IRQ.

If you can get away from taxing the CPU like that altogether, that would be even better.

Davide Ancri

Re: nanospin_*() functions and SMP

Post by Davide Ancri » Mon Jul 23, 2007 6:26 am

Mario Charest wrote:
nanospin() doesn't use rdtsc -

Ah!!! For some reason I assumed it was.
Me too ;-)

So, this is a good piece of news for me: thread free to migrate and
inline delay still working, great!

Davide

--
/* Ancri Davide - <davidea at prisma-eng dot it> */

Robert Craig

Re: nanospin_*() functions and SMP

Post by Robert Craig » Mon Jul 23, 2007 5:24 pm

Evan Hillas wrote:
I'd say disable interrupts. It's the only way to be sure. ;) The other way is to setup a separate hardware timer attached to an IRQ.

If you can get away from taxing the CPU like that altogether, that would be even better.
Have to be very careful about interrupts and SMP.l.. When you turn off
interrupts, you're turning them off on the CPU that you're running on
which might not be the same as the one where the actual ISR will be
caught (depending upon architecture and BSP)....

Evan Hillas

Re: nanospin_*() functions and SMP

Post by Evan Hillas » Tue Jul 24, 2007 2:30 am

Robert Craig wrote:
Have to be very careful about interrupts and SMP.l.. When you turn off
interrupts, you're turning them off on the CPU that you're running on
which might not be the same as the one where the actual ISR will be
caught (depending upon architecture and BSP)....
I know. Davide just wants to keep nanospin consistent in one thread is all. He's not interested in interrupts. Colin indicated that nanospin exists primarily for drivers to do precise timing with their hardware with interrupts already disabled. I just extended that to include all uses of nanospin.

If there is other threads that can access the same data then use locks I guess. If there is other ISRs that can access the same data then maybe a redesign is in order.


Evan

Post Reply

Return to “qnx.rtos”