I ran into an issue that might be procedural, but I though you guys might want to know anyway.
We are pursuing with IBM HW support as of 2013-07-18.
I am going to test further in my lab. I suspect this may be related to
bkprofata and rstprofdata copying over some internal seed for MAC addresses
I plan to try this in my lab on p5 rackmount servers via an HMC.
If I can reproduce it there, then I expect support’s response to be “don’t do that”.
As such, I also will try a factory reset to see if that will clear the condition.
If I cannot reproduce it there, then it’s either SDMC/FSM related (which is going away),
or it’s blade/Flex Node related (No other test resources, but maybe L3 can help).
If L3 decides that rstprofdata cannot be used on a different system,
then I would want them to A) Limit the command to that functionality,
and B) Update documentation for both commands to reflect this.
### BEGIN NOTICE ###
bkprofdata & rstprofdata were used to clone the LPAR layout from one blade to another.
To reset the WWNs, I was able to delete and re-add the virtual fibre adapters.
New LPARs and new virtual fibre adapters automatically get WWNs with the blade/node number as part of the WWN.
This part works as I would expect.
To reset the MAC addresses, this did not work.
Delete and re-add virtual ethernet adapters does not change the MAC addresses.
Adding a new adapter that did not exist before to the same slot number,
on the same LPAR ID, on two different Flex nodes, and both get the same MAC accress.
Current resolution is to override the MAC address with a user specified value in the LPAR profile.
This can be done from Profile -> Virtual -> Ethernet -> Advanced -> checkbox
Change from commandline:
chsyscfg -m Server-7895-23X-SN1012345 -r prof -i \
'name=DefaultProfile,lpar_id=4,"virtual_eth_adapters=""2/0/1//0/1/ETHERNET0/DEADBEEF0402/all/none"""'
To remove and Readd:
chsyscfg -m Server-7895-23X-SN1012345 -r prof -i \
'name=DefaultProfile,lpar_id=4,"virtual_eth_adapters-=""2/0/1//0/1/ETHERNET0//all/none"""'
chsyscfg -m Server-7895-23X-SN1012345 -r prof -i \
'name=DefaultProfile,lpar_id=4,"virtual_eth_adapters+=""2/0/1//0/1/ETHERNET0/DEADBEEF0402/all/none"""'
NOTES:
I’ve never seen this happen on any other POWER series servers, and I’ve built a lot of p7 systems, ranging from p710 to p780, including matching LPARs between CECs. This is on top of the whole slew of LPARable systems I’ve built and/or supported.
I looked into the profile data backup files themselves, and there is no mention of system serial, system name, WWN prefix, or MAC prefix.
I restored mode 3 of the profile data backups prior to any config work, and when adding new virtual NICs to LPARs, the MAC addresses still mirror eachother.
I plan to test this with two p505 systems on an HMC to see if similar issues occur.
I don’t have the resources to test this on blades, or on another SDMC.
We are pursuing with IBM HW support as of 2013-07-18
### END NOTICE ###
After a week, still no no response from support,
but I think I found out why this was a problem.
On physical hardware, “lssyscfg -r lpar” will show virtual_eth_mac_base_value=
On the flex nodes, this value is not exposed.
I can’t tell if this is an SDMC/FSM limitation, or a flex node limitation.
I know that IVM sees it, but am not sure about HMC.
So, when LPAR profiles are copied over, they will bring the VEMBV,
and there is no way to change it short of deleting and re-creating.
All in all, it may just be easier to use mksyscfg from the start.
An example might be:
mksyscfg -r lpar -m Server-8205-E6D-SN10FFFFF -i profile_name=DefaultProfile,\
name=production,lpar_id=4,lpar_env=aixlinux,min_mem=1024,desired_mem=8192,\
max_mem=16384,mem_mode=ded,proc_mode=shared,min_proc_units=0.1,\
desired_proc_units=0.1,max_proc_units=8.0,min_procs=1,desired_procs=1,\
max_procs=8,sharing_mode=uncap,uncap_weight=8,max_virtual_slots=10,\
\"virtual_scsi_adapters=2/client/1/vioserver/24/1\",\
\"virtual_eth_adapters=8/0/25//0/1,9/0/863//0/1\",auto_start=0
But there’s already reference online for this sort of command.
Also, while working on a p740 via IVM, I ran into more differences from HMC/SDMC.
When you add a client LPAR with virtual SCSI, IVM automagically creates the VIO server virtual scsi server adapter. In addition, +1 from that slot it creates a virtual serial adapter for mkvterm.
If you’re used to adding virtual scsi adapters in order, and you don’t skip a slot on the mksyscfg lines, then you’ll get this error:
[VIOSE01050173-0290] Cannot create virtual serial adapter in the management partition in the virtual slot number specified 20.
I couldn’t find this error anywhere else on the internet, and it was a little confusing since I wasn’t making a virtual serial adapter.
