June 6/06 AAReport

CERT Drill 06/03/2006

Contributors:
Brendan Mitchell K6BMK Fred Brown W5BN Mitch Mitchell K6BK
Robert Swain KA4JSR Ted Hamm W6GMQ Howard White KY6LA

Brendan Mitchell K6BMK (age 12)

What I learned at the drill today working as the EOC Scribe:
Make sure you always bring the equipment you need and the little adapters that you need to hook everything up. We spent a lot of time messing around with wires and plugs, and you know the laptop computer settings are hard to understand for getting packet working even if the connections are right. We had about 6 radios going at the same time and it was confusing enough without temporary wires all over the table, and the radio room was all piled up with storage stuff before we even started. Our main radio didn't work at first because it didn't even have an antenna attached. The radios were all on 2 Meters and sounded terrible when anybody transmitted, so you had to have headphones and grab the volume knob. The drill was hours longer than we expected and we had to go get some snacks and juice.
You always need to have a comm log and be ready to write in it. Not complicated, just a single sheet of paper with columns for message numbers and times it was sent and cleared, so you can keep track of all messages fast and accurately. I never Scribed before and when they said building numbers it was hard to know if you got it right, so some of the adults listened too and helped me check it. Even they asked for repeats on the numbers it was given so fast. Make sure your messages are cleared when the activity on it is really done, and make sure your message actually gets through and is received (bomb squad message never sent by CERT1 then they asked EOC for a reply!). If you mess up and don't have a message logged right it could be about a person who is hurt and needs first aid or is lost and could die.
Have a comm plan that everybody knows about and it is a bad idea changing it at the last minute unless you really have to. Have already planned things for going on other channels somebody has tried out. Don't add lots of drill problems like knocking out your main comm repeater with a fake earthquake just for fun, because that is not the real purpose of the drill which is to send messages and it just makes everyone confused and wastes lots of time trying things that don't work. It took more than a half hour following CERT1 around on 5 different channels, one after the other some on simplex and other repeaters too that did not work, ending up back on the Coronado repeater again, forgetting about the earthquake I guess. It was a new Kenwood radio and nobody knew how to set the dial except it was similar to my Dad's old one so he got it on the channel right away, otherwise we would have been reading the Kenwood manual and wondering if it was set right each time. CERT1's lo-power HTs were 3.5 miles away (we measured it on Google Earth later) and nothing else worked but the Coronado repeater, but meanwhile we could have been getting the packet radio on the air with Winlink instead.
I did do 8 messages in the comm log and that was very satisfying knowing how to mark them cleared when they were done. This was a good drill for having that many messages come from CERT1, and also from Transport1 which was taking the CERT guys the antennas and masts that they forgot. He would have been lost there at the drill if he didn't have his radio on.
I like these drills that have tactical actions in them instead of just replacing a fax machine doing messages on forms. Only I think they still need a signature to know if it is from the right person in charge, and I didn't get any signatures from CERT1, so maybe they were the ones in charge but should have told us that if there were other people there sending messages too.

Brendan Mitchell K6BMK (age 12)
Back to top.

Fred Brown W5BN

1. What went Right - Great participation and response. Message handling was efficient and accurate.
2. What went wrong - Because the EOC was cluttered and full of chairs, tables and other storage debris the staging of equipment and the operational status of the comms equipment was compromised. the equipment should be in a ready to go status, not disassembled components locked up in cabinets.
3. What I liked - The Drill scenario was excellent and gave excellent training to the CERT and Cero participants, the drill allowed several adaptations to develop which was excellent.
4. What I disliked - The drill length was expected to last till 10 but turned out to be till 2. It would be reasonable to know the planned time ahead of time.
5. What changes - I would like to contact the mayor and ask if there is a way to have exclusive use of the EOC storage closet for Communications support to the EOC. I would hope that the council would approve this and we would have a secure comms room.
6. What I learned - The comms log was excellent and should be used as our primary tracking tool. The ICS 213 form in hindsight is the best form to use since DHS groups use and recognize it. The CERO team was adaptive to the obstacles encountered and performed with exceptional skill.

Fred Brown W5BN
CERO/CERT
Back to top.

Mitch Mitchell K6BK

Here is a quote from the ARRL ARECC Level 2 Unit 17... "Beware setting too many smaller goals for one exercise, unless the goal is to create confusion and high levels of stress. This is not recommended for a drill situation in most cases, unless your volunteers have been prepared for it and warned well in advance. Volunteers who will accept high stress in a real emergency may not be willing to do so during a drill, and you might lose some members."
In the drill today most members of the staff were tinkering with packet hardware and configuration settings for Winlink, when the first change of the frequency plan was given by IC (with no advanced notice?) and a small disagreement followed over the freq choice, being a favorite DXer channel. Even if this freq change had the purpose of performing simplex testing, with the low-power HTs at IC it was pretty much a waste of time and extra work (for more than 30 minutes) for EOC at a busy time trying to get the Winlink working.
EOC agreed to try on simplex but at the time was not aware IC was using only an HT on low power. Depending on the type of radio low power can be various settings, for example the VX-7R has 3 low power choices of 2.5, 1.0, and 0.05 watts. On the FT-60R, which is more typical, medium power is 2.0 and low power is 0.5 watts. IC had large concrete/steel buildings directly in their path, had no elevation advantage (except the one successful test that was done when they ran up to the 3rd floor of the building) and was using a duckie antenna. EOC running 50 watts with a rooftop gain antenna gave IC a false sense of communications effectiveness, since EOC was always full quieting. I don't think anyone knew the actual distance, which was later measured on Google Earth to be 3.5 miles.
So the first choice (the DXer freq) gave very bad copy from IC to EOC, of course EOC with 50 watts and a roof antenna was full quieting at IC, so they always copied the instructions for another change. So these were tried: another 2 meter simplex channel, a UHF simplex freq that was busy, a clear UHF simplex channel... still no good, and then IC admitted having only the HT on low power, so EOC suggested a UHF repeater on Mt Otay (2 Meters was being used at the drill site by other tactical teams... with potential for Rx desense) which was no good either... just too far away.
IC insisting on using low power to conserve the battery. Knowing all these factors ahead of time any experienced ham would not expect simplex to work. Why not have an extra battery handy and use maximum power? Or at least switch to 5 watts for a moment during the freq changes to make that process of evaluation faster. There were many repeats of reports on each channel change as IC tried slightly different positions. Or IC could make a quick estimate of battery usage... the 2 drills were expected to be about 3 hours long total (actual message handling) with a Tx/Rx duty cycle of maybe 20 to 1. If IC could have monitored their HT battery voltage as most modern radios do, they would have found that a NiMh or Li-Ion battery would have been fine at 5 watts for the whole drill even without an extra battery. I think IC was overly pessimistic in its choice of staying on low power, even when EOC asked them to increase power. A spare radio (which IC should have available anyway in case one is dropped or mis-programmed) is also another battery.
The comm plan changes interrupted work flow at EOC getting the Winlink operational. Also when eventually back on the Coronado repeater IC's low-power HT was still marginal copy at times, dropping out, depending on their position, as it had been at the beginning of the drill. If the comm plan had been reviewed for alternates, both repeaters and simplex, with known minimum acceptable power for each, the freq change would have been successful and taken less than a minute. If EOC had other radios in operation the spontaneous comm plan changes could have also been incompatible with those other operations. The reason I stress this issue of the comm plan, is that of all the uncertainties of an emergency or a drill, the comm plan is one thing that can be studied with alternates evaluated, and tried out in advance experimentally for the best results later.

Subject: When repeaters fail.

I was reminded this morning during the ARES North District net (since the NCS announced it), and actually Fred mentioned it yesterday too, that.....
When repeaters fail in an emergency the standard action is for stations to transmit on the output freq of the repeater, not go seeking another simplex channel. This action is very simple, press one button to cancel the freq shift setting.
The important point is that all the other stations responding to the emergency will be listening on the repeater output freq, expecting it to be operational, rather than some unknown simplex freq. If the repeater comes back with emergency power or whatever, the net is already there on simplex in progress, just enable the shift and continue with no actual QSY required.
We all know how many stations get left behind when a large net is asked to QSY.... if the new freq is bad (busy or has interference) another QSY will lose even more people... because either they don't hear the instructions, the new freqs are not in memory, or hams forget how to dial in their radios.
So you need to ask why the standard method was not used in the drill, instead of hopping around 6 times... I am not exaggerating, we QSYed 6 times trying to get anything to work, ending up back on the Coronado repeater. Imagine an experiment with a net of 25 people, and then QSY 6 times, assume losing say a conservative one fifth of them each freq change... there would only be 6 stations left.
Even if there were fewer lost, think of all the wasted time and risk of losing ALL the stations from one mistake, such as NCS setting their radio wrong. It is very revealing how dangerous QSY is for a net... if conditions were more difficult and starting with 25 stations, you lost say one third of them each QSY for 6 times, there would only be 2 left!

_____________________
K6BK bravo kilo Mitch
K6VS Vacation Station
WAS,WAZ,IOTA,DXCC/QRP
Scripps Ranch CA DM12
Back to top.

Robert Swain KA4JSR

The Coronado Emergency Radio Operators (CERO) members met at Coronado EOC at 07:45 to get the EOC readied and load the truck to go to the incident site. Key items were loaded in quick time while others familiarized themselves with EOC procedures and a newly installed dual-band radio. We loaded the gasoline, the generator, emergency kit #1, 50’ coax, clipboard with message forms and capabilities forms in Ted’s vehicle. We each had our “go kits” with food, water and snacks to get us through the first few hours of operations. Ted, W6GMQ and Robert, KA4JSR soon after, departed for the incident site.
On arrival at the incident site, we parked and located the rest of the group and learned that there were going to be two drills; a morning one for “team A” and an afternoon one for “team B”. Each team had about 100 people. All Coronado people, CERT and the two CERO members were assigned to team B. While team A was doing the actual drill, team B would be doing learning centers which consisted of cribbing training, fire containment practice among other learning experiences
Drill management had each of the two teams decide who would be their Incident Commander. After some discussion, they came to agreement on who would be the leader of each team. We (CERO) made ourselves known to both ICs along with what our capabilities would be. Within 15 minutes, the teams were all called to the training center bleachers and told the details of the drill. In both cases for teams A and B, it was a massive Earthquake with lots of damage and injuries.
Soon after the briefing, Ted and Robert stepped outside and all of the available hams from several CERT teams assembled around us (the red vests really stick out in a crowd of green). Very quickly we assigned a local frequency for tactical communications and assigned a different frequency for communication out of the incident area to the Coronado EOC. Everyone was given their roles in support of the first drill (team A) and we set off to set up our equipment in the shaded area against one of the buildings. It was all going so well...
While Ted positioned the vehicle with the equipment, Robert contacted the Coronado EOC via radio and passed on as much information as was known at the time about the impending drill. Along with the basic information about our knowledge up to that point, Robert also put into effect the pre-planned outage of the Coronado Repeater, 147.18 MHz.
Meanwhile, Ted was unloading the equipment and trying to start the Honda 1KW generator. Hondas are extremely reliable and easy to start, but not this time. The generator simply would not even fire. After hundreds of pulls on the starter cord, it just wasn’t going to start. This caused a significant delay in determining that we were about to have an even worse morning.
Messages started to flow soon after the start time. Robert copied messages from “Command” and passed them on to the Coronado EOC on the established backup simplex frequency, 446.0 MHz UHF simplex. Thanks to the Coronado Fire Dept, we purchased the brand new UHF radio and temporarily installed it on the EOC the day before the drill. The approaching black cloud over our communications plan was rapidly approaching as the Coronado EOC couldn’t hear the incident site team on any UHF simplex, repeater, or VHF simplex frequency we tried. The cause was that we were operating with Robert’s handheld radio and he was determined to stay on low power (0.5 watts) to conserve battery power. This caused him to leave the radio command area and walk up two flight of stairs to get enough height to make contact with the EOC directly. After much consternation on the parts of both the EOC and the incident site teams, it was discovered that we had forgotten to load the mast sections and the antenna in the truck before we left the EOC first thing in the morning. Without an antenna, kit 1 was out of business for packet, voice, WinLink, and we didn’t even have an external antenna for the hand-held. To maintain reliable communications, the incident site needed to run high power on the handhelds which rapidly drains internal batteries. We were stuck in a growingly ugly situation.
In desperation, we had to ‘repair’ the artificially ‘failed’ Coronado repeater, and ask the EOC to send a truck over with the mast sections and the antenna. Had this been an actual event, this would not have been a realistic request. A sharp and stinging lesson was learned about double checking the packing list before you leave the EOC.
The Coronado repeater worked very well and provided adequate coverage for the incident site except when in the RF shadow against the buildings (exactly like we were). Another lesson learned on placement and thoughts about the direction of the RF radiation.
Soon after the call to the EOC, Jim brought over the antenna and mast sections. The first drill was over shortly afterwards. Two hours time had elapsed from start time to the point we realized we didn’t have an antenna and then actually got one in hand. Although we were able to send and receive all 7 messages via voice radio, it was never pretty, and it required more time and effort than it should have on all radio participants.
We stopped for lunch and team B had their briefing before they started out to the damaged buildings. Ted was selected to be the IC’s shadow for the second drill. This was both a blessing and curse as it left Robert handling the tactical radio and the Coronado communications bridge at the same time. The traffic flow was such that it didn’t turn out to be a problem.
Robert tried to establish WinLink communications using the newly assembled 20’ mast and antenna on emergency kit 1. Although a message was successfully sent from our location to Howard’s Blackberry, we never had real confidence that we had reliable communications via WinLink. We need more time and training to increase our confidence and kill a few tenacious bugs.
Team B’s drill went much smoother than team A’s, as one would expect. We were working with Coronado CERT members, Ted was the shadow, the IC was a ham (in CERT mode) and we had reliable voice communications between Ted and Robert, and Robert and the Coronado EOC. Messages flow was light but all messages were passed quickly and accurately. Simulations provided by the EOC CERO personnel were realistic and timely. Follow-up questions were on the mark, as they were questions that would be normally asked during an actual emergency; clarifications, repeat requests, and follow-ups. This drill was very professional and very realistic.
The second drill ended around 1350 local time with all participants ready to call it a day.
What we did right:

We had good people at the EOC and the incident site.
There was always someone answering the radio when the incident site called the EOC.
The EOC personnel were helpful in providing advice, suggestions, and solving problems that the incident site was having.

All radio equipment worked well.

Operators on both ends were able to change frequencies quickly and accurately across the two bands we now have available.

The impromptu organization of available hams at the incident site was close to perfect and very realistic.

Nets were suggested, reviewed and implemented in seconds.

Tactical call signs were established quickly and FCC rules were respected.

What we did wrong:

We failed to load all of the equipment for the incident site in the truck.

The new message form has a few challenges that still need to be resolved.

There’s not enough space to write more than a line of message text.

The organization of the fields seems to be random. It needs to flow more like a conversation.

We were using Ted’s vehicle as the communications position for outgoing messages. We could have used his high-powered mobile rig installed in the vehicle but nobody thought of that at the time.

We positioned ourselves in the RF shadow of the building, making things harder than they needed to be.

We didn’t check the generator before we needed it.

We didn’t leave enough time to get familiar with WinLink before we tried to use it.

We didn’t establish a digital message connection (packet or WinLink)
Lessons learned:

We need a checklist to ensure we get all of the equipment loaded.

Check all equipment regularly.

Stay familiar with the “VFO Mode” on your radios – you’ll need it in an actual emergency.
I’d like to add this addendum to the draft report I’ve already circulated.

The reason for failing the Coronado repeater was to create a realistic hardship that would likely follow an actual earthquake. It seems reasonable that one of our own primary communication paths (our repeater) would be damaged or otherwise out of service when we need it.
The scenario that I planned as part of the drill planning team involved the loss of the repeater early into the drill. This would cause the CERO team to either use the output of the repeater as a simplex frequency or force a new frequency all together. Since we were simulating the failure of the repeater, moving to a new frequency made the most sense. Considering that we were operating at the local incident scene using 2 meters, we thought we would use our new 440 MHz rig in the EOC to communicate with a dual-band handheld at the scene. After several attempts on several frequencies, it was no longer productive to the drill to continue with the failure scenario of the repeater.
There comes a point in training where the lesson has been learned and you need to move on. A simulation can go too far and damage morale and other lessons that could have been learned if you carry them on too long. I came up with the repeater failure scenario; we suffered with it and learned that without our external antenna and high-power transmitters, and our poor choice of location against the building, we were having severe difficulties without the repeater.
The lesson was learned and I reinstated the repeater (as if it had been repaired or the back-up generator had been started) so that we could continue learning other lessons. Neither the failure nor the repair of the repeater, were random events. The simulation had the desired effect. It stirred thought and emotion and will drive us to a better answer next time we have a repeater failure.

Respectfully,

Robert Swain KA4JSR
Back to top.

Ted Hamm W6GMQ

This was a San Diego County wide drill at the Naval Training Center. This report covers my assignment as Shadow to the Coronado CERT Team Leader, Kevin at the Incident Command Post. Normally we would have the CERO Incident Command Post communications link to the Coronado EOC located at the Incident Command Post, but with the failure of our gas generator we established a relay location next to building 89 with AC power.

Overview

I set up my Shadow of Kevin the Coronado CERT Team Leader in a grassy area in the shade. I used my Icom T2H on 146.43 MHz Simplex, low power, to communicate to Robert KA4JSR acting as the Coronado CERO Incident Command Post and relay to the Coronado EOC. I handled five messages ( IC-1 through IC-5 ) over the two hour drill, as given to me verbally by the CERT Team Leader. I had a smooth cooperative working relation with Kevin. At the Incident Command Post the CERT Team Leader had on his staff Search & Rescue Leader, Fire Suppression Leader, Medical Leader, Logistics Leader & CERO communications ( his shadow ).
The Drill All messages were verbal an example of the verbal messages I received from the CERT Team Leader :
2 victims, Delayed
1 victim, Immediate
2 ambulances requested

I filled out a message form with the above information then requested its priority which was stated Routine. I filled in the message #, date & time. On one message I questioned the message priority of Routine since there was an Immediate Victim.
The message was relayed to my CERO IC communication relay.
The message was then relayed to the Coronado EOC.
I received word back that the message was received at the Coronado EOC, and when the Ambulances were dispatched, which I reported back to the CERT Team Leader Kevin.

What worked well.

Good working relations with the Coronado CERT Team Leader.
Good communications to the CERO Command Post, KA4JSR.
Good responses back from the Coronado EOC through the CERO Command Post.
The Simplex channel 146.430 MHz was a clear channel.
The work pace was slow since I handled 6 messages in 2 hours, probably realistic.

Improvements suggested.

It is clear that the message generation from the CERT field teams needs training. There should be a common message form that everybody follows, see enclosed message form suggestion. The field team leader at the incident site for the Search & Rescue, Medical, Fire Suppression , or Logistics --- should originate the message. Their name must be listed as the originator so if clarification is needed one can go back to the source.

Message Example:

The message originates at the Incident Site by the Search & Rescue team leader. Using their Family Radio the message is sent to CERT Team Leader's command post Search & Rescue staff member then it is verbally given to the CERT Team Leader. Then the CERT Team Leader verbally gives his CERO Shadow the message. Information is lost/changed with the number of people it verbally goes through. The CERT teams use no formal message structure or form and all messages are verbal. Using a standardized message form would help the message originator to fill in all the blanks for a meaningful message. The message should go from the CERT Team Leaders Staff to the CERT Team Leaders CERO's Shadow where the formal message is then given to the CERT Team Leader for approval to Transmit to the EOC.

If distance or obstructions causes a communications drop out for the Family Radio then the CERO Ham radio may be called in to assist.
A CERO equipment check list is required so that in the heat of the battle items are not overlooked ! I overlooked the field site antenna and mast. Thanks to Jim N6RSL he brought them over to the Incident site from the Coronado EOC.
When gathering the equipment for Portable 1 or 2, I suggest, a check list, then lay all the equipment out on the EOC floor to take full account. Have a second person double check the equipment, then load it in the vehicle.
Three of us tried to get the gas generator going, not even a cough. It may be the fuel is from the Nov. 03 Cedar fire time. Jim N6RSL has taken the generator to work on it. With new fuel we should add the fuel stabilizer to counter the fuel aging problem. A tool kit with spark plug wrench should accompany the generator.
We should get the names of the Hams on the other CERT teams and invite them to come to Coronado and train with us.

Ted Hamm W6GMQ
Back to top.

Howard White KY6LA

Synopsis:

1. What Went Right

a. CERO COMM LOG FORMS Worked better extremely well
b. ICS rules allowed for Field Team with IC to set Comm Channel
c. Message Numbers used
d. Messages cleared by numbers
e. UHF worked from EOC

2. What Went Wrong

a. Comm Plan ignored or unknown in the field
b. Did not fall back to repeater output frequency for backup communications
c. Simplex did not work
d. Did not understand that field had no antenna for an hour
e. Overly complex scenario with Repeater failure
f. Some confusion at EOC as to who sets Comm Plan
g. ICS -Form 213 should be used?
h. Comm Log did not have signatures
i. EOC is a mess
j. Portable units are a mess
k. Hot wire on the EOC data line - no solder
l. Tools hard to find or not labeled
m. EOC message form far too complex to use…we ignored it.
n. Winlink effectively failed due to the inability to establish reliable packet links
o. EOC Antennas are far too close - resulted in only 1 usable VHF channel
p. Winlink/packet overloaded all other channels…
q. Not set up Winlink before drill
r. Did not read manuals for new radio before the drill

3. What I liked

a. EOC and Field worked well together
b. EOC and/or Field cleared every message
c. We tried Winlink
d. We did not lose any messages
e. Brendan invented the signature block

4. What I disliked

a. Drill scenario did not give enough hams practice
b. The physical condition of the EOC and the equipment
c. Complex Message Forms
d. Disregard of the original Comm Plan.

5. What I would change

a. Drill Scenario to include more ham participation
b. Use ICS-213
c. Add signature block to Comm Log
d. Establish Comm Plan Decision Protocols

Howard White KY6LA
Back to top.

Brendan Mitchell K6BMK (age 12)
What I learned at the drill today working as the EOC Scribe: Make sure you always bring the equipment you need and the little adapters that you need to hook everything up. We spent a lot of time messing around with wires and plugs, and you know the laptop computer settings are hard to understand for getting packet working even if the connections are right. We had about 6 radios going at the same time and it was confusing enough without temporary wires all over the table, and the radio room was all piled up with storage stuff before we even started. Our main radio didn't work at first because it didn't even have an antenna attached. The radios were all on 2 Meters and sounded terrible when anybody transmitted, so you had to have headphones and grab the volume knob. The drill was hours longer than we expected and we had to go get some snacks and juice. You always need to have a comm log and be ready to write in it. Not complicated, just a single sheet of paper with columns for message numbers and times it was sent and cleared, so you can keep track of all messages fast and accurately. I never Scribed before and when they said building numbers it was hard to know if you got it right, so some of the adults listened too and helped me check it. Even they asked for repeats on the numbers it was given so fast. Make sure your messages are cleared when the activity on it is really done, and make sure your message actually gets through and is received (bomb squad message never sent by CERT1 then they asked EOC for a reply!). If you mess up and don't have a message logged right it could be about a person who is hurt and needs first aid or is lost and could die. Have a comm plan that everybody knows about and it is a bad idea changing it at the last minute unless you really have to. Have already planned things for going on other channels somebody has tried out. Don't add lots of drill problems like knocking out your main comm repeater with a fake earthquake just for fun, because that is not the real purpose of the drill which is to send messages and it just makes everyone confused and wastes lots of time trying things that don't work. It took more than a half hour following CERT1 around on 5 different channels, one after the other some on simplex and other repeaters too that did not work, ending up back on the Coronado repeater again, forgetting about the earthquake I guess. It was a new Kenwood radio and nobody knew how to set the dial except it was similar to my Dad's old one so he got it on the channel right away, otherwise we would have been reading the Kenwood manual and wondering if it was set right each time. CERT1's lo-power HTs were 3.5 miles away (we measured it on Google Earth later) and nothing else worked but the Coronado repeater, but meanwhile we could have been getting the packet radio on the air with Winlink instead. I did do 8 messages in the comm log and that was very satisfying knowing how to mark them cleared when they were done. This was a good drill for having that many messages come from CERT1, and also from Transport1 which was taking the CERT guys the antennas and masts that they forgot. He would have been lost there at the drill if he didn't have his radio on. I like these drills that have tactical actions in them instead of just replacing a fax machine doing messages on forms. Only I think they still need a signature to know if it is from the right person in charge, and I didn't get any signatures from CERT1, so maybe they were the ones in charge but should have told us that if there were other people there sending messages too. Brendan Mitchell K6BMK (age 12) Back to top.

Fred Brown W5BN
1. What went Right - Great participation and response. Message handling was efficient and accurate. 2. What went wrong - Because the EOC was cluttered and full of chairs, tables and other storage debris the staging of equipment and the operational status of the comms equipment was compromised. the equipment should be in a ready to go status, not disassembled components locked up in cabinets. 3. What I liked - The Drill scenario was excellent and gave excellent training to the CERT and Cero participants, the drill allowed several adaptations to develop which was excellent. 4. What I disliked - The drill length was expected to last till 10 but turned out to be till 2. It would be reasonable to know the planned time ahead of time. 5. What changes - I would like to contact the mayor and ask if there is a way to have exclusive use of the EOC storage closet for Communications support to the EOC. I would hope that the council would approve this and we would have a secure comms room. 6. What I learned - The comms log was excellent and should be used as our primary tracking tool. The ICS 213 form in hindsight is the best form to use since DHS groups use and recognize it. The CERO team was adaptive to the obstacles encountered and performed with exceptional skill. Fred Brown W5BN CERO/CERT Back to top.

Mitch Mitchell K6BK
Here is a quote from the ARRL ARECC Level 2 Unit 17... "Beware setting too many smaller goals for one exercise, unless the goal is to create confusion and high levels of stress. This is not recommended for a drill situation in most cases, unless your volunteers have been prepared for it and warned well in advance. Volunteers who will accept high stress in a real emergency may not be willing to do so during a drill, and you might lose some members." In the drill today most members of the staff were tinkering with packet hardware and configuration settings for Winlink, when the first change of the frequency plan was given by IC (with no advanced notice?) and a small disagreement followed over the freq choice, being a favorite DXer channel. Even if this freq change had the purpose of performing simplex testing, with the low-power HTs at IC it was pretty much a waste of time and extra work (for more than 30 minutes) for EOC at a busy time trying to get the Winlink working. EOC agreed to try on simplex but at the time was not aware IC was using only an HT on low power. Depending on the type of radio low power can be various settings, for example the VX-7R has 3 low power choices of 2.5, 1.0, and 0.05 watts. On the FT-60R, which is more typical, medium power is 2.0 and low power is 0.5 watts. IC had large concrete/steel buildings directly in their path, had no elevation advantage (except the one successful test that was done when they ran up to the 3rd floor of the building) and was using a duckie antenna. EOC running 50 watts with a rooftop gain antenna gave IC a false sense of communications effectiveness, since EOC was always full quieting. I don't think anyone knew the actual distance, which was later measured on Google Earth to be 3.5 miles. So the first choice (the DXer freq) gave very bad copy from IC to EOC, of course EOC with 50 watts and a roof antenna was full quieting at IC, so they always copied the instructions for another change. So these were tried: another 2 meter simplex channel, a UHF simplex freq that was busy, a clear UHF simplex channel... still no good, and then IC admitted having only the HT on low power, so EOC suggested a UHF repeater on Mt Otay (2 Meters was being used at the drill site by other tactical teams... with potential for Rx desense) which was no good either... just too far away. IC insisting on using low power to conserve the battery. Knowing all these factors ahead of time any experienced ham would not expect simplex to work. Why not have an extra battery handy and use maximum power? Or at least switch to 5 watts for a moment during the freq changes to make that process of evaluation faster. There were many repeats of reports on each channel change as IC tried slightly different positions. Or IC could make a quick estimate of battery usage... the 2 drills were expected to be about 3 hours long total (actual message handling) with a Tx/Rx duty cycle of maybe 20 to 1. If IC could have monitored their HT battery voltage as most modern radios do, they would have found that a NiMh or Li-Ion battery would have been fine at 5 watts for the whole drill even without an extra battery. I think IC was overly pessimistic in its choice of staying on low power, even when EOC asked them to increase power. A spare radio (which IC should have available anyway in case one is dropped or mis-programmed) is also another battery. The comm plan changes interrupted work flow at EOC getting the Winlink operational. Also when eventually back on the Coronado repeater IC's low-power HT was still marginal copy at times, dropping out, depending on their position, as it had been at the beginning of the drill. If the comm plan had been reviewed for alternates, both repeaters and simplex, with known minimum acceptable power for each, the freq change would have been successful and taken less than a minute. If EOC had other radios in operation the spontaneous comm plan changes could have also been incompatible with those other operations. The reason I stress this issue of the comm plan, is that of all the uncertainties of an emergency or a drill, the comm plan is one thing that can be studied with alternates evaluated, and tried out in advance experimentally for the best results later. Subject: When repeaters fail. I was reminded this morning during the ARES North District net (since the NCS announced it), and actually Fred mentioned it yesterday too, that..... When repeaters fail in an emergency the standard action is for stations to transmit on the output freq of the repeater, not go seeking another simplex channel. This action is very simple, press one button to cancel the freq shift setting. The important point is that all the other stations responding to the emergency will be listening on the repeater output freq, expecting it to be operational, rather than some unknown simplex freq. If the repeater comes back with emergency power or whatever, the net is already there on simplex in progress, just enable the shift and continue with no actual QSY required. We all know how many stations get left behind when a large net is asked to QSY.... if the new freq is bad (busy or has interference) another QSY will lose even more people... because either they don't hear the instructions, the new freqs are not in memory, or hams forget how to dial in their radios. So you need to ask why the standard method was not used in the drill, instead of hopping around 6 times... I am not exaggerating, we QSYed 6 times trying to get anything to work, ending up back on the Coronado repeater. Imagine an experiment with a net of 25 people, and then QSY 6 times, assume losing say a conservative one fifth of them each freq change... there would only be 6 stations left. Even if there were fewer lost, think of all the wasted time and risk of losing ALL the stations from one mistake, such as NCS setting their radio wrong. It is very revealing how dangerous QSY is for a net... if conditions were more difficult and starting with 25 stations, you lost say one third of them each QSY for 6 times, there would only be 2 left! _____________________ K6BK bravo kilo Mitch K6VS Vacation Station WAS,WAZ,IOTA,DXCC/QRP Scripps Ranch CA DM12 Back to top.

Robert Swain KA4JSR
The Coronado Emergency Radio Operators (CERO) members met at Coronado EOC at 07:45 to get the EOC readied and load the truck to go to the incident site. Key items were loaded in quick time while others familiarized themselves with EOC procedures and a newly installed dual-band radio. We loaded the gasoline, the generator, emergency kit #1, 50’ coax, clipboard with message forms and capabilities forms in Ted’s vehicle. We each had our “go kits” with food, water and snacks to get us through the first few hours of operations. Ted, W6GMQ and Robert, KA4JSR soon after, departed for the incident site. On arrival at the incident site, we parked and located the rest of the group and learned that there were going to be two drills; a morning one for “team A” and an afternoon one for “team B”. Each team had about 100 people. All Coronado people, CERT and the two CERO members were assigned to team B. While team A was doing the actual drill, team B would be doing learning centers which consisted of cribbing training, fire containment practice among other learning experiences Drill management had each of the two teams decide who would be their Incident Commander. After some discussion, they came to agreement on who would be the leader of each team. We (CERO) made ourselves known to both ICs along with what our capabilities would be. Within 15 minutes, the teams were all called to the training center bleachers and told the details of the drill. In both cases for teams A and B, it was a massive Earthquake with lots of damage and injuries. Soon after the briefing, Ted and Robert stepped outside and all of the available hams from several CERT teams assembled around us (the red vests really stick out in a crowd of green). Very quickly we assigned a local frequency for tactical communications and assigned a different frequency for communication out of the incident area to the Coronado EOC. Everyone was given their roles in support of the first drill (team A) and we set off to set up our equipment in the shaded area against one of the buildings. It was all going so well... While Ted positioned the vehicle with the equipment, Robert contacted the Coronado EOC via radio and passed on as much information as was known at the time about the impending drill. Along with the basic information about our knowledge up to that point, Robert also put into effect the pre-planned outage of the Coronado Repeater, 147.18 MHz. Meanwhile, Ted was unloading the equipment and trying to start the Honda 1KW generator. Hondas are extremely reliable and easy to start, but not this time. The generator simply would not even fire. After hundreds of pulls on the starter cord, it just wasn’t going to start. This caused a significant delay in determining that we were about to have an even worse morning. Messages started to flow soon after the start time. Robert copied messages from “Command” and passed them on to the Coronado EOC on the established backup simplex frequency, 446.0 MHz UHF simplex. Thanks to the Coronado Fire Dept, we purchased the brand new UHF radio and temporarily installed it on the EOC the day before the drill. The approaching black cloud over our communications plan was rapidly approaching as the Coronado EOC couldn’t hear the incident site team on any UHF simplex, repeater, or VHF simplex frequency we tried. The cause was that we were operating with Robert’s handheld radio and he was determined to stay on low power (0.5 watts) to conserve battery power. This caused him to leave the radio command area and walk up two flight of stairs to get enough height to make contact with the EOC directly. After much consternation on the parts of both the EOC and the incident site teams, it was discovered that we had forgotten to load the mast sections and the antenna in the truck before we left the EOC first thing in the morning. Without an antenna, kit 1 was out of business for packet, voice, WinLink, and we didn’t even have an external antenna for the hand-held. To maintain reliable communications, the incident site needed to run high power on the handhelds which rapidly drains internal batteries. We were stuck in a growingly ugly situation. In desperation, we had to ‘repair’ the artificially ‘failed’ Coronado repeater, and ask the EOC to send a truck over with the mast sections and the antenna. Had this been an actual event, this would not have been a realistic request. A sharp and stinging lesson was learned about double checking the packing list before you leave the EOC. The Coronado repeater worked very well and provided adequate coverage for the incident site except when in the RF shadow against the buildings (exactly like we were). Another lesson learned on placement and thoughts about the direction of the RF radiation. Soon after the call to the EOC, Jim brought over the antenna and mast sections. The first drill was over shortly afterwards. Two hours time had elapsed from start time to the point we realized we didn’t have an antenna and then actually got one in hand. Although we were able to send and receive all 7 messages via voice radio, it was never pretty, and it required more time and effort than it should have on all radio participants. We stopped for lunch and team B had their briefing before they started out to the damaged buildings. Ted was selected to be the IC’s shadow for the second drill. This was both a blessing and curse as it left Robert handling the tactical radio and the Coronado communications bridge at the same time. The traffic flow was such that it didn’t turn out to be a problem. Robert tried to establish WinLink communications using the newly assembled 20’ mast and antenna on emergency kit 1. Although a message was successfully sent from our location to Howard’s Blackberry, we never had real confidence that we had reliable communications via WinLink. We need more time and training to increase our confidence and kill a few tenacious bugs. Team B’s drill went much smoother than team A’s, as one would expect. We were working with Coronado CERT members, Ted was the shadow, the IC was a ham (in CERT mode) and we had reliable voice communications between Ted and Robert, and Robert and the Coronado EOC. Messages flow was light but all messages were passed quickly and accurately. Simulations provided by the EOC CERO personnel were realistic and timely. Follow-up questions were on the mark, as they were questions that would be normally asked during an actual emergency; clarifications, repeat requests, and follow-ups. This drill was very professional and very realistic. The second drill ended around 1350 local time with all participants ready to call it a day. What we did right: We had good people at the EOC and the incident site. There was always someone answering the radio when the incident site called the EOC. The EOC personnel were helpful in providing advice, suggestions, and solving problems that the incident site was having. All radio equipment worked well. Operators on both ends were able to change frequencies quickly and accurately across the two bands we now have available. The impromptu organization of available hams at the incident site was close to perfect and very realistic. Nets were suggested, reviewed and implemented in seconds. Tactical call signs were established quickly and FCC rules were respected. What we did wrong: We failed to load all of the equipment for the incident site in the truck. The new message form has a few challenges that still need to be resolved. There’s not enough space to write more than a line of message text. The organization of the fields seems to be random. It needs to flow more like a conversation. We were using Ted’s vehicle as the communications position for outgoing messages. We could have used his high-powered mobile rig installed in the vehicle but nobody thought of that at the time. We positioned ourselves in the RF shadow of the building, making things harder than they needed to be. We didn’t check the generator before we needed it. We didn’t leave enough time to get familiar with WinLink before we tried to use it. We didn’t establish a digital message connection (packet or WinLink) Lessons learned: We need a checklist to ensure we get all of the equipment loaded. Check all equipment regularly. Stay familiar with the “VFO Mode” on your radios – you’ll need it in an actual emergency. I’d like to add this addendum to the draft report I’ve already circulated. The reason for failing the Coronado repeater was to create a realistic hardship that would likely follow an actual earthquake. It seems reasonable that one of our own primary communication paths (our repeater) would be damaged or otherwise out of service when we need it. The scenario that I planned as part of the drill planning team involved the loss of the repeater early into the drill. This would cause the CERO team to either use the output of the repeater as a simplex frequency or force a new frequency all together. Since we were simulating the failure of the repeater, moving to a new frequency made the most sense. Considering that we were operating at the local incident scene using 2 meters, we thought we would use our new 440 MHz rig in the EOC to communicate with a dual-band handheld at the scene. After several attempts on several frequencies, it was no longer productive to the drill to continue with the failure scenario of the repeater. There comes a point in training where the lesson has been learned and you need to move on. A simulation can go too far and damage morale and other lessons that could have been learned if you carry them on too long. I came up with the repeater failure scenario; we suffered with it and learned that without our external antenna and high-power transmitters, and our poor choice of location against the building, we were having severe difficulties without the repeater. The lesson was learned and I reinstated the repeater (as if it had been repaired or the back-up generator had been started) so that we could continue learning other lessons. Neither the failure nor the repair of the repeater, were random events. The simulation had the desired effect. It stirred thought and emotion and will drive us to a better answer next time we have a repeater failure. Respectfully, Robert Swain KA4JSR Back to top.

Ted Hamm W6GMQ
This was a San Diego County wide drill at the Naval Training Center. This report covers my assignment as Shadow to the Coronado CERT Team Leader, Kevin at the Incident Command Post. Normally we would have the CERO Incident Command Post communications link to the Coronado EOC located at the Incident Command Post, but with the failure of our gas generator we established a relay location next to building 89 with AC power. Overview I set up my Shadow of Kevin the Coronado CERT Team Leader in a grassy area in the shade. I used my Icom T2H on 146.43 MHz Simplex, low power, to communicate to Robert KA4JSR acting as the Coronado CERO Incident Command Post and relay to the Coronado EOC. I handled five messages ( IC-1 through IC-5 ) over the two hour drill, as given to me verbally by the CERT Team Leader. I had a smooth cooperative working relation with Kevin. At the Incident Command Post the CERT Team Leader had on his staff Search & Rescue Leader, Fire Suppression Leader, Medical Leader, Logistics Leader & CERO communications ( his shadow ). The Drill All messages were verbal an example of the verbal messages I received from the CERT Team Leader : 2 victims, Delayed 1 victim, Immediate 2 ambulances requested I filled out a message form with the above information then requested its priority which was stated Routine. I filled in the message #, date & time. On one message I questioned the message priority of Routine since there was an Immediate Victim. The message was relayed to my CERO IC communication relay. The message was then relayed to the Coronado EOC. I received word back that the message was received at the Coronado EOC, and when the Ambulances were dispatched, which I reported back to the CERT Team Leader Kevin. What worked well. Good working relations with the Coronado CERT Team Leader. Good communications to the CERO Command Post, KA4JSR. Good responses back from the Coronado EOC through the CERO Command Post. The Simplex channel 146.430 MHz was a clear channel. The work pace was slow since I handled 6 messages in 2 hours, probably realistic. Improvements suggested. It is clear that the message generation from the CERT field teams needs training. There should be a common message form that everybody follows, see enclosed message form suggestion. The field team leader at the incident site for the Search & Rescue, Medical, Fire Suppression , or Logistics --- should originate the message. Their name must be listed as the originator so if clarification is needed one can go back to the source. Message Example: The message originates at the Incident Site by the Search & Rescue team leader. Using their Family Radio the message is sent to CERT Team Leader's command post Search & Rescue staff member then it is verbally given to the CERT Team Leader. Then the CERT Team Leader verbally gives his CERO Shadow the message. Information is lost/changed with the number of people it verbally goes through. The CERT teams use no formal message structure or form and all messages are verbal. Using a standardized message form would help the message originator to fill in all the blanks for a meaningful message. The message should go from the CERT Team Leaders Staff to the CERT Team Leaders CERO's Shadow where the formal message is then given to the CERT Team Leader for approval to Transmit to the EOC. If distance or obstructions causes a communications drop out for the Family Radio then the CERO Ham radio may be called in to assist. A CERO equipment check list is required so that in the heat of the battle items are not overlooked ! I overlooked the field site antenna and mast. Thanks to Jim N6RSL he brought them over to the Incident site from the Coronado EOC. When gathering the equipment for Portable 1 or 2, I suggest, a check list, then lay all the equipment out on the EOC floor to take full account. Have a second person double check the equipment, then load it in the vehicle. Three of us tried to get the gas generator going, not even a cough. It may be the fuel is from the Nov. 03 Cedar fire time. Jim N6RSL has taken the generator to work on it. With new fuel we should add the fuel stabilizer to counter the fuel aging problem. A tool kit with spark plug wrench should accompany the generator. We should get the names of the Hams on the other CERT teams and invite them to come to Coronado and train with us. Ted Hamm W6GMQ Back to top.

Howard White KY6LA
Synopsis: 1. What Went Right a. CERO COMM LOG FORMS Worked better extremely well b. ICS rules allowed for Field Team with IC to set Comm Channel c. Message Numbers used d. Messages cleared by numbers e. UHF worked from EOC 2. What Went Wrong a. Comm Plan ignored or unknown in the field b. Did not fall back to repeater output frequency for backup communications c. Simplex did not work d. Did not understand that field had no antenna for an hour e. Overly complex scenario with Repeater failure f. Some confusion at EOC as to who sets Comm Plan g. ICS -Form 213 should be used? h. Comm Log did not have signatures i. EOC is a mess j. Portable units are a mess k. Hot wire on the EOC data line - no solder l. Tools hard to find or not labeled m. EOC message form far too complex to use…we ignored it. n. Winlink effectively failed due to the inability to establish reliable packet links o. EOC Antennas are far too close - resulted in only 1 usable VHF channel p. Winlink/packet overloaded all other channels… q. Not set up Winlink before drill r. Did not read manuals for new radio before the drill 3. What I liked a. EOC and Field worked well together b. EOC and/or Field cleared every message c. We tried Winlink d. We did not lose any messages e. Brendan invented the signature block 4. What I disliked a. Drill scenario did not give enough hams practice b. The physical condition of the EOC and the equipment c. Complex Message Forms d. Disregard of the original Comm Plan. 5. What I would change a. Drill Scenario to include more ham participation b. Use ICS-213 c. Add signature block to Comm Log d. Establish Comm Plan Decision Protocols Howard White KY6LA Back to top.

Return to After Action Reports.

Return to CERO Inc. Home Page.

Please send comments & corrections to Frank:
Last Updated: 06/20/2006