spfile used in RAC 10.2.0.4

We have a 3-node 10.2.0.4 RAC on Solaris 10. One node crashed due to ORA-600 [kturacf1] error. The main DBA of this database was out, so I had to bring it up.

I can see in the alert log of that instance the DBA had run a command to change a parameter in the scope of spfile (alter system set db_file_name_convert='+DG1,+DG1
' scope=spfile) sometime ago.
When I tried a bring up the instance with "srvctl start instance" command, in the alert log I can see that changed parameter db_file_name_convert takes the new value and is different from what the value is in the other 2 nodes. So that discrepancy prevents the instance from being open. In alert log, it also shows spfile is set to the common spfile on ASM.

My question #1 is how did srvctl find this spfile file?. In the $ORACLE_HOME/dbs directory, there is only 1 init file. This init file does NOT point to the common spfile, which exists on ASM. So it seems to me srvctl does not go to $ORACLE_HOME/dbs to look for spfile/pfile, rather it seems to "remember" where the spfile was when the instance was up last time. Or it looks at the other 2 nodes and finds out where the command spfile is?

I ended up using sqlplus to start the instance with the local init file. Now I have one instance running with pfile, 2 other instances running with spfile. And the parameters match fine currently. So db_file_name_convert is unset. However at the next bounce, the other 2 nodes will try to take on the new value for db_file_name_convert, and the discrepancy will prevent them from starting up.
So my question #2 is if there is a rolling way changing this parameter or all non-system modifiable parameters. It doesn't seem possible to change it without shutting all 3 nodes down then bringing them up. Is it so?

Thank you,
Chen

--
http://www.freelists.org/webpage/oracle-l