Program in external SDRAM and caching

hi,
I have a simple program for my AT91RM9200 to turn led on and off. When I
run it from internal SRAM it works ok, but when I copy it into external
SDRAM and execute from there it jumps to address 0x2000001c (FIQ vector or
is it just a coincidence?) instead of 0x2000003c (start code is there).
64MB of SDRAM was tested and looks ok. However when I disable instruction
cache, this program works ok in SDRAM but is slow...
Is there a problem with my SDRAM initialization?
If I should configure cache for SDRAM, how can I do it?

regards,
Kuba

odi wrote:

> I have a simple program for my AT91RM9200 to turn led on and off. When I
> run it from internal SRAM it works ok, but when I copy it into external
> SDRAM and execute from there it jumps to address 0x2000001c (FIQ vector or
> is it just a coincidence?) instead of 0x2000003c (start code is there).
> 64MB of SDRAM was tested and looks ok. However when I disable instruction
> cache, this program works ok in SDRAM but is slow...
> Is there a problem with my SDRAM initialization?
> If I should configure cache for SDRAM, how can I do it?

How does your board look like? Which clock do you use for the SDRAM?
Recently in a project for which I work there were a problem with the SDRAM,
because the capacity of the address traces were too high (it has a low
specified limit, depends on the clock, see the datasheet for details). We
measured the traces on a PCB (another one, not populated) and it were too
high for 133 MHz. Works fine now for 100 MHz SDRAM clock.

-- 
Frank Buss, fb@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de

>odi wrote:
>
>> I have a simple program for my AT91RM9200 to turn led on and off. When
I
>> run it from internal SRAM it works ok, but when I copy it into
external
>> SDRAM and execute from there it jumps to address 0x2000001c (FIQ vector
or
>> is it just a coincidence?) instead of 0x2000003c (start code is
there).
>> 64MB of SDRAM was tested and looks ok. However when I disable
instruction
>> cache, this program works ok in SDRAM but is slow...
>> Is there a problem with my SDRAM initialization?
>> If I should configure cache for SDRAM, how can I do it?
>
>How does your board look like? Which clock do you use for the SDRAM?
>Recently in a project for which I work there were a problem with the
SDRAM,
>because the capacity of the address traces were too high (it has a low
>specified limit, depends on the clock, see the datasheet for details).
We
>measured the traces on a PCB (another one, not populated) and it were
too
>high for 133 MHz. Works fine now for 100 MHz SDRAM clock.
>
>-- 
>Frank Buss, fb@frank-buss.de
>http://www.frank-buss.de, http://www.it4-systems.de
>
It is a board designed by me, unfortunately it is 2-layer PCB but SDRAM
chips are placed as close to CPU as I could place them. I was worried by it
but as far as I've tested it the data is stored and read back correctly. I
use 60MHz clock for SDRAM.
Strange thing occurs when I debug this application with JTAG - when
application jumps to wrong address, I check value in 0x20000000 (reset
vector of new application) and it has a correct value.

odi wrote:

> It is a board designed by me, unfortunately it is 2-layer PCB but SDRAM
> chips are placed as close to CPU as I could place them. I was worried by it
> but as far as I've tested it the data is stored and read back correctly. I
> use 60MHz clock for SDRAM.

Ok, maybe then it is not a problem. For the 9G20 the maximum allowed
capacity is 10 pF for 133 MHz, but for 66 MHz looks like it can be 50 pF,
maybe for your chip, too. Maybe there are problems with the power supply:
Could be difficult for a 2-layer PCB to place all necessary capacitors with
good connection to ground.

But if you can write and read the data of the SDRAM by software and it
looks ok, maybe it is just a software problem. But did you test all data
and address lines? There are some sophisticated algorithms to do this. E.g.
a simple one for testing the address lines of a SRAM is to write something
to address 0, then write something different to address 2^n and then
testing again address 0. You can do this for n=1..number of address lines.
Of course, for SDRAM with RAS and CAS it would be more difficult.

For free support you can try http://support.atmel.no/bin/customer
Sometimes they have some helpful answers :-)

-- 
Frank Buss, fb@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de

>But if you can write and read the data of the SDRAM by software and it
>looks ok, maybe it is just a software problem. But did you test all data
>and address lines? There are some sophisticated algorithms to do this.
E.g.
>a simple one for testing the address lines of a SRAM is to write
something
>to address 0, then write something different to address 2^n and then
>testing again address 0. You can do this for n=1..number of address
lines.
>Of course, for SDRAM with RAS and CAS it would be more difficult.

hi
Thanks for the reply:) My memory test writes a pattern to each byte of
64MB memory and then reads it back. I tested patterns like
0x00000000,0x11111111,0x33333333,0x55555555,0x0f0f0f0f,0x00ff00ff and their
negated values. Test succeeded each time I ran it, single write/read
operation takes about 6 seconds, so it has a throughput of ~20MB/s.
I suppose that there is something wrong with cache... I don't know much
about either it or MMU, but I'm still reading about it. If someone has any
idea what could be wrong I would really appreciate it:)
regards,
Kuba

odi wrote:

> Thanks for the reply:) My memory test writes a pattern to each byte of
> 64MB memory and then reads it back. I tested patterns like
> 0x00000000,0x11111111,0x33333333,0x55555555,0x0f0f0f0f,0x00ff00ff and their
> negated values. 

Do you write the test pattern to all bytes of the memory for one test? Then
you don't see bugs in the address lines. I've found a nice summary of some
test patterns:

http://digitalelectronics.blogspot.com/2009/01/memories-test-patterns-algorithms-part_12.html

I think another test algorithm could be to use a pseudo random number
generator with some seed x for filling the memory, then set the seed x
again and testing the memory.

> I suppose that there is something wrong with cache... I don't know much
> about either it or MMU, but I'm still reading about it. If someone has any
> idea what could be wrong I would really appreciate it:)

If you use the standard bootstrap from Atmel (you can change this, too),
MMU should be disabled and all memory access is routed immediatly and with
flat adressing model to the address bus.

So this could be a problem with the hardware, because if you enable the
cache, I think there will be more burst transfers than single memory
access. At least for our problem with the 133 MHz / 100 MHz SDRAM clock it
occured frequently with Linux and much random traffic on the memory
interface, but not in u-boot with disabled caches. But we were able to
reproduce it even in u-boot with dummy PDC transfers from SSC and a
parallel loop for reading/writing to another memory interface when it was
running at 133 MHz. Don't know, if the AT91RM9200 has PDC.

-- 
Frank Buss, fb@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de

>Do you write the test pattern to all bytes of the memory for one test?
Then
>you don't see bugs in the address lines. I've found a nice summary of
some
>test patterns:
>
>http://digitalelectronics.blogspot.com/2009/01/memories-test-patterns-algorithms-part_12.html

actually I write test pattern X to first word in memory, value X+1, to 2nd
word, and so on, so there should be no bugs in address lines.. I'll try
other tests when I'll have more time

>If you use the standard bootstrap from Atmel (you can change this, too),
>MMU should be disabled and all memory access is routed immediatly and
with
>flat adressing model to the address bus.
>
>So this could be a problem with the hardware, because if you enable the
>cache, I think there will be more burst transfers than single memory
>access. At least for our problem with the 133 MHz / 100 MHz SDRAM clock
it
>occured frequently with Linux and much random traffic on the memory
>interface, but not in u-boot with disabled caches. But we were able to
>reproduce it even in u-boot with dummy PDC transfers from SSC and a
>parallel loop for reading/writing to another memory interface when it
was
>running at 133 MHz. Don't know, if the AT91RM9200 has PDC.

Now I see that it could really be a problem with hardware. I'll try
lowering bus speed and see what happens then.
Thanks for your help:)

If your chip has separate data and instruction caches, bytes that you
write out to RAM may not be the same bytes you execute from there -
you have to flush the data cache out to memory, then invalidate the
instruction cache, to make sure the right data gets through.

hi;)
I've tested SDRAM memory with march and address algorithms from the site
given by Frank and I haven't seen a single error. I've also written GALPat
test but tested only first kilobyte (which took me about 40minutes @60MHz -
whole memory would be scanned for about 5years if my assumptions are
correct:D). So I was pretty sure that memory is OK, yet I tried lowering
bus speed to ~5MHz but it didn't work.
Then I tried some things with cache - no results.
Fortunately I've read on one of ARM forums that cache and SDRAM have
"radically different timing properties" so I tried using 180MHz processor
clock and 90MHz bus clock (previously it was 60MHz) and everything started
working correctly;)

I would never have guessed such a stupid thing by myself, I hate
hardware:P

thanks for your help guys;)

regards,
Kuba