Tale From The Trenches: The Debug Of Damocles

My good friend and colleague Rich Stroffolino (@MrAnthropology) is collecting Tales from the Trenches about times when we did things that we didn’t expect to cause problems. I wanted to share one of my own here about the time I knocked a school offline with a debug command.

I Got Your Number

The setup for this is pretty simple. I was deploying a CallManager setup for a multi-site school system deployment. I was using local gateways at every site to hook up fax lines and fire alarms with FXS/FXO ports for those systems to dial out. Everything else got backhauled to a voice gateway at the high school with a PRI running MGCP.

I was trying to figure out why the station IDs that were being send by the sites weren’t going out over caller ID. Everything was showing up as the high school number. I needed to figure out what was being sent. I was at the middle school location across town and trying to debug via telnet. I logged into the router and figured I would make a change, dial my cell phone from the VoIP phone next to me, and see what happened. Simple troubleshooting, right?

I did just that. My cell phone got the wrong caller ID. And my debug command didn’t show me anything. So I figured I must be debugging the wrong thing. Not debug isdn q931 after all. Maybe it’s a problem with MGCP. I’ll check that. But I just need to make sure I’m getting everything. I’ll just debug it all.

debug mgcp packet detail

Can You Hear Me Now?

Veterans of voice are probably screaming at me right now. For those who don’t know, debug anything detail generates a ton of messages. And I’m not consoled into the router. I’m remote. And I didn’t realize how big of a problem that was until my console started scrolling at 100 miles an hour. And then froze.

Turns out, when you overwhelm a router CPU with debug messages, it shuts off the telnet window. It also shuts off the console as well, but I wouldn’t have known that because I was way far way from that port. But I did starting hearing people down the hall saying, “Hello? Hey, are you still there? Weird, it just went dead.”

Guess what else a router isn’t doing when it’s processing a tidal wave of debug messages? It’s not processing calls. At all. For five school sites. I looked down at my watch. It was 2:00pm. That’s bad. Elementary schools get a ton of phone calls within the last hour of being in session. Parents calling to tell kids to wait in a pickup line or ride a certain bus home. Parents wanting to check kids out early. All kinds of things. That need phones.

I raced out of my back room. I acknowledged the receptionists comment about the phones not working. I jumped in my car and raced across town to the high school. I managed not to break any speed limits, but I also didn’t loiter one bit. I jumped out of my car and raced into the building. The look on my face must have warded off any comments about phone system issues because no one stopped me before I got to the physical location of the voice gateway.

I knew things were bad. I didn’t have time to console in and remove the debug command. I did what ever good CCIE has been taught since the beginning of time when they need to remove a bad configuration that broke their entire lab.

I pulled the power cable and cycled the whole thing.

I was already neck deep in it. It would have taken me at least five minutes to get my laptop ready and consoled in. In hindsight, that would have been five wasted minutes since the MGCP debugger would have locked out the console anyway. As the router was coming back up, I nervously looked at the terminal screen for a login prompt. Now that the debugger wasn’t running, everything looked normal. I waiting impatiently for the MGCP process to register with CallManager once more.

I kept repeating the same status CLI command while I refreshed the gateway page in CallManager over and over. After a few more tense minutes, everything was back to normal. I picked up a phone next to the rack and dialed my cell phone. It rang. I was happy. I walked back to the main high school office and told them that everything was back to normal.

Tom’s Take

My post-mortem was simple. I did dumb things. I shouldn’t have debugged remotely. I shouldn’t have used the detail keyword for something so simple. In fact, watching my screen fill up with five sites worth of phone calls in a fraction of a second told me there was too much going on behind the scenes for me to comprehend anyway.

That was the last time I ever debugged anything in detail. I made sure from that point forward to start out small and then build from there to find my answers. I also made sure that I did all my debugging from the console and not a remote access window. And the next couple of times I did it were always outside of production hours with a hand ready to yank the power cable just in case.

I didn’t save the day. At best, all I did was cover for my mistake. If it had been a support call center or a hospital I probably would have been fired for it. I made a bad decision and managed to get back to operational without costing money or safety.

Remember when you’re doing your job that you need to keep an eye on how your job will affect everything else. We don’t troubleshoot in a vacuum after all.

Advertisements

The Voice of SD-WAN

SD-WAN is about migrating your legacy hardware away from silos like MPLS and policy-based routing and instead integrating everything under one dashboard and one central location to make changes and see the impacts that those changes have. But there’s one thing that SD-WAN can’t really do yet. And that’s prepare us the for the end of TDM voice.

Can You Hear Me Now?

Voice is a way of life for some people. Cisco spent years upon years selling CallManager into every office they could. From small two-line shops to global organizations with multiple PRIs and TEHO configured everywhere. It was a Cisco staple for years. Which also had Avaya following along quickly to get into the act too.

Today’s voice world is a little less clear. Millenials hate talking on the phone. Video is an oddity when it comes to communications. Asynchronous chat programs like WhatsApp or Slack rule the day today. People would rather communicate via text than voice. We all have mobile devices and the phone may be one of the least used apps on it.

Where does that leave traditional voice services? Not in a good place for sure. We still need phone lines for service-focused businesses or when we need to call a hotline for support. But the office phone system isn’t getting any new features anytime soon. The phone system is like the fax machine in the corner. It’s a feature complete system that is used when it has to be used by people that are forced to use it unhappily.

Voice systems are going to stay where they are by virtue of their ubiquity. They exist because TDM technology hasn’t really advanced in the past 20 years. We still have twisted pair connections to deliver FXO lines. We still have the most basic system in place to offer services to our potential customers and users. I know this personally because when I finally traded out my home phone setup for a “VoIP” offering from my cable provider, it was really just an FXS port on the back of a residential cable modem. That’s as high-tech as it gets. TDM is a solved problem.

Call If You WANt To

So, how does SD-WAN play into this? Well, as it turns out, SD-WAN is replacing the edge router very quickly. Devices that used to be Cisco ISRs are now becoming SD-WAN edge devices. They aggregate WAN connections and balance between them. They take MPLS and broadband and LTE instead of serial and other long-forgotten connection methods.

But you know what SD-WAN appliances can’t aggregate? TDM lines. They don’t have cards that can accept FXO, FXS, or even PRI lines. They don’t have a way to provide for DSP add-in cards or even come with onboard transcoding resources. There is no way for an SD-WAN edge appliance to function as anything other than a very advanced packet router.

This is a good thing for SD-WAN companies. It means that they have a focused, purpose built device that has more software features than hardware muscle. SD-WAN should be all about data packets. It’s not a multitool box. Even the SD-WAN vendors that ship their appliances with LTE cards aren’t trying to turn them into voice routers. They’re just easing the transition for people that want LTE backup for data paths.

Voice devices were moved out of the TDM station and shelf and into data routers as Cisco and other companies tried to champion voice over IP. We’re seeing the fallout from those decisions today. As the data routing devices become more specialized and focused on the software aspects of the technology, the hardware pieces that the ISR platform specialized in are now becoming a yoke holding the platform back. Now, those devices are causing those platforms to fail to evolve.

I can remember when I was first thinking about studying for my CCIE Voice lab back in 2007-2008. At the time, the voice lab still have a Catalyst 6500 switch running in it that needed to be configured. It had a single T1 interface on a line card that you had to get up and running in CallManager. The catch? That line card would only work with a certain Supervisor engine that only ran CatOS. So, you have to be intimately familiar with CatOS in order to run that lab. I decided that it wasn’t for me right then and there.

Hardware can hold the software back. ISRs can’t operate voice interfaces in SD-WAN mode. You can’t get all the advanced features of the software until you pare the hardware down to the bare minimum needed to route data packets. If you need to have the router function as a TDM aggregator or an SBC/IPIPGW you realize that the router really should be dedicated to that purpose. Because it’s functioning more as a TDM platform than a packet router at that point.


Tom’s Take

The world of voice that I lived in five or six years ago is gone. It’s been replaced with texting and Slack/Spark/WebEx Teams. Voice is dying. Cell phones connect us more than we’ve ever been before but yet we don’t want to talk to each other. That means that the rows and rows of desk phones we used to use are falling by the wayside. And so too are the routers that used to power them. Now, we’re replacing those routers with SD-WAN devices. And when the time finally comes for use to replace those TDM devices, what will we use? That future is very murky indeed.

Slacking Off

A Candlestick Phone (image courtesy of WIkipedia)

A Candlestick Phone (image courtesy of WIkipedia)

There’s a great piece today on how Slack is causing disruption in people’s work habits. Slack is a program that has dedicated itself to getting rid of email, yet we now find ourselves mired in Slack team after Slack team. I believe the real issue isn’t with Slack but instead with the way that our brains are wired to handle communication.

Interrupt Driven

People get interrupted all the time. It’s a fact of life if you work in business, not just IT. Even if you have your head down typing away at a keyboard and you’ve closed out all other forms of distraction, a pop up from an email or a ringing or vibrating phone will jar your concentration out of the groove and force your brain to deal with this new intruder into your solitude.

That’s evolution working against you. When we were hunters and gatherers our brain had to learn how to deal with external threats when we were focused on a task like stalking a mammoth or looking for sprouts on the forest floor. Our eyes are even developed to take advantage of this. Your peripheral vision will pick up movement first, followed by color, then finally it can discern the shape of an object. So when your email notifier slides out from the system tray or notification window it triggers your primitive need to address the situation.

In the modern world we don’t hunt mammoths or forage for shoots any longer. Instead, our survival instinct has been replaced by the need to answer communications as fast as possible. At first it was returning phone calls before the end of the day. Then it became answering emails expediently. That changed into sending an immediate email response that you had seen the email and were working on a response. Then came instant messaging for corporate environments and the idea of “presence”, which allows everyone to know what you’re doing and when you’re doing it. Which has led us to ever presence – the idea that we’re never really not available.

Think about the last time you saw someone was marked unavailable in a chat window and you sent the message anyway. Perhaps you thought they would see the message the next time they logged in or returned to their terminal. Or perhaps you guessed that they had set their status as away to avoid distraction. Either way, the first thought you had was that this person wasn’t really gone and was available.

Instant messaging programs like Slack bridge the gap because synchronous communications channels like phone calls and asynchronous channels like email. In the past, we could deal with phone calls because it required the attention of both parties involved. A single channel was opened and it was very apparent that you were holding a conversation, at least until the invention of call waiting. On the other hand, email is asynchronous by nature because we can put all of our thoughts down in a single message over the course of minutes or even hours and send it into the void. Reliable delivery ensures that it will make it to the destination but we don’t know when it will be read. We don’t know when the response will come or in what form. The receiving party may not even read your message!

The Need to Please

Think back to the last time you responded to an email. How often did you start your response with “Sorry for the delay” or some version of that phrase? In today’s society, we’ve become accustomed to instant responses to things. Amy Lewis (@CommsNinja) is famous for having an escalation process for reaching her:

  1. Text message
  2. Twitter DM
  3. Email
  4. Phone Call
  5. Anything else
  6. Voice mail

She prefers instant communication and rapid response. In a lot of cases, this is very crucial. If you need an answer to a question quickly there are ways to reach people for immediate reply. But the desire to have immediate response for all forms of communication is a bit much.

Our brains don’t help us in this matter. When we get an email or a communication from someone, we feel compelled to respond to it. It’s like a checkbox that needs to be checked. And so we will drop everything else to work on a reply even if it means we’re displeasing someone for a short time to please someone immediately.

Many of the time management systems that have been created to deal with massive email flows, such as GTD are centered on the idea of dealing with things as the come in and pigeonholing them until they can be dealt with appropriately. By treating everything the same you disappoint everyone equally until everything can evaluated. There are cutouts for high priority communications, but the methods themselves tell you to keep those exceptions small and rare so as not to disrupt the flow of things.

The key to having coherent and meaningful conversations with other people is the same online as it is in person. Rather than speaking before you think, you should take the time to consider your thoughts and respond with appropriately measured words. It’s easier to do this via email since there is built-in delay but it works just the same in instant message conversations as well. An extra minute of thought won’t make someone angry with you, but not taking that extra minute could make someone very cross with you down the road.


Tom’s Take

I agree with people that say Slack is great for small teams spread everywhere to help categorize thoughts and keep projects on track. It takes away the need for a lot of status update emails and digests of communications. It won’t entirely replace email for communications and it shouldn’t be seen that way. Instead, the important thing to realize about programs like Slack is that they will start pushing your response style more toward quick replies with little information. You will need to make a conscious decision to push back a bit to make measured responses to things with more information and less response for the sake of responding. When you do you’ll find that instant messaging tools augment your communications instead of complicating them.

My Thoughts On The Death Of IP Telephony

A Candlestick Phone (image courtesy of WIkipedia)

A Candlestick Phone (image courtesy of Wikipedia)

Greg Ferro (@EtherealMind) posted a thought provoking article about collaboration in his Human Infrastructure magazine (which you should be reading). He talks about the death of IP Telephony and the rise of asynchronous communications methods like Slack. He’s got a very interesting point of view. I just happen to disagree with a few of his assertions.

IP Telephony Is Only Mostly Dead

Greg’s stance that IP Telephony is dead is a bit pointed to say the least. He is correct that the market isn’t growing. It is also true that a great number of new workers entering the workforce prefer to use their smartphones for communications, especially the asynchronous kind. However, desk phones are a huge part of corporate communications going forward.

IT shops have a stilted and bizarre world view. If you have a workforce that has to be mobile, whether it be for making service calls or going to customer sites for visits, you have a disproportionately large number of mobile users for sure. But what about organizations that don’t have large mobile populations? What about financial firms or law offices or hospitals? What about retail organizations? These businesses have specific needs for communications, especially with external customers and users.

Imagine if your pharmacy replaced their phone with a chat system? How about your doctor’s office throwing out their PBX and going to an email-only system? How would you feel about that? A couple of you might cheer because they finally “get it”. But a large number of people, especially more traditional non-technical folks, would switch providers or move their business elsewhere. That’s because some organizations rely on voice communications. For every millennial dumping their office phone to use a mobile device there is still someone they need to call on the other end.

We’re not even talking about the important infrastructure that still needs a lot of specialized communications gear. Fax machines are still a huge part of healthcare and legal work. Interactive Voice Response (IVR) systems are still crucial to handle call volumes for things like support lines. These functions can’t be replaced on mobile devices easily. You can fake IVRs and call queuing with the right setup, but faxing things to a mobile device isn’t possible.

Yes, services do exist to capture fax information as a TIFF or JPG and email it to the destination. But for healthcare and legal, this breaks confidentiality clauses and other important legal structures. The area around secure faxing via email is still a bit murky, and most of the searches you can do for the topic revolve around companies trying to tell you that it’s acceptable and okay to use (as long as you use their product).

IP Telephony isn’t far removed from buggy whip manufacturers. The oft-cited example of a cottage industry has relevance here. At some point after the introduction of the automobile, buggy whip growth slowed and eventually halted. But they didn’t go away permanently. The market did contract and still exists to this day. It’s not as big as the 13,000-strong market it once was, but it does exist today to meet a need that people still have. Likewise, IP Telephony will still have solutions to meet needs for specific customers. Perhaps we’ll contract down to two or three providers at some point in the future, but it will never really go away.

I’ll Have My People IM Your People

Contacting people is an exercise today. There are so many ways to reach someone via various communications that you have to spend time figuring out how to reach them. Direct message, Text message, Phone call, Voice Mail, Email, and smoke signals are all valid communications forms. It is true that a lot of these communications are moving toward asynchronous methods. But as mentioned above, a lot of customer-facing businesses are still voice-only.

Sales is one of these areas that is driven by sound. The best way to sell something to someone is to do it face-to-face. But a phone call is a close second. Sales works because you can use your voice to influence buyers. You can share their elation and allay their fears. You can be soothing or exciting or any range of emotion in between. That’s something you don’t get through the cold text of instant messaging or email.

It’s also much harder to ignore phone calls. Sure, you can send read receipts with emails but these are rarely implemented and even more rarely used correctly in my experience. Phone calls alert people about intent. Even ignoring or delaying them means being send to a voice mail box or to another phone in the department where your call can be dealt with. The synchronous nature of the communication means there has to be a connection with someone. You can’t just launch bytes of text into the ether and hope it gets where it’s supposed to go.

It is true that these voice communications happen via mobile numbers more often than not in this day and age. But corporations still prefer those calls to go through some kind of enterprise voice system. Those systems can track communications and be audited. Records can be subpoenaed for legal reasons without needing to involve carriers.

It’s much easier for call centers to track productivity via phone logs than seeing who is on the phone. If you’ve ever worked in a corporate call center, you know there are metrics for everything you do on the phone. Average call time, average wait time, amount of non-call time, and so on. Each of these metrics can be tracked via a desk phone with a headset, not so with a mobile phone and an app.


Tom’s Take

I live on my mobile phone. I send emails and social media updates. I talk in Slack and Skype and other instant messaging platforms. But I still get on the phone at least three times a week to talk to someone. Most of those calls take place on a conference bridge. That’s because people want to hear someone’s voice. It’s still comforting and important to listen to someone.

Doing away with IP Telephony sounds like an interesting strategy for small businesses and startups. It’s a cost-reduction method that has benefits in the short term. But as companies grow and change they will soon find that having a centralized voice system to control and manipulate calls is a necessity. Given the changes in voice technology in the last few years, I highly expect that “centralized” voice will eventually be a pay-per-seat cloud leased model with specific executives and support personnel using traditional phones while non-critical employees have no voice communications device or choose to use their personal mobile device.

IP Telephony isn’t dead. It’s not even dying. But it’s well past the age where it needs to consider retirement and a long and fulfilling life concentrating on specific people instead of trying to make everyone happy.

 

 

Gathering No MOS

mossBall1

If you work in the voice or video world, you’ve undoubtedly heard about Mean Opinion Scores (MOS). MOS is a rough way of ranking the quality of the sound on a call. It’s widely used to determine the over experience for the user on the other end of the phone. MOS represents something important in the grand scheme of communications. However, MOS is quickly becoming a crutch that needs some explanation.

That’s Just Like Your Opinion

The first think to keep in mind when you look at MOS data is that the second word in the term is opinion. Originally, MOS was derived by having selected people listen to calls and rank them on a scale of 1 (I can’t hear you) to 5 (We’re sitting next to each other). The idea was to see if listeners could distinguish when certain aspects of the call were changed, such as pathing or exchange equipment. It was an all-or-nothing ranking. Good calls got a 4 or even rarely a 5. Most terrible calls got 2 or 3. You take the average of all your subjects and that gives your the overall MOS for your system.

voip-qualitypbx

When digital systems came along, MOS took on an entirely different meaning. Rather than being used to subjectively rank call quality, MOS became a yardstick for tweaking the codec used to digitally transform analog speech to digital packets. Since this has to happen in order for the data to be sent, all digital calls must have a codec somewhere. The first codecs were trying to approximate the quality of a long distance phone call, which was the gold standard for quality. After that target was reached, providers started messing around the codecs in question to reduce bandwidth usage.

G.711 is considered the baseline level of call quality from which all others are measure. It has a relative MOS of 4.1, which means very good voice quality. It also uses around 64 kbps of bandwidth. As developers started playing with encoding schemes and other factors, they started developing codecs which used significantly less bandwidth and had almost equivalent quality. G.729 uses only 8 kbps of bandwidth but has a MOS of 3.9. It’s almost as good as G.711 in most cases but uses an eighth of the resources.

MOS has always been subjective. That was until VoIP system providers found that certain network metrics have an impact on the quality of a call. Things like packet loss, delay, and jitter all have negative impacts on call quality. By measuring these values a system could give an approximation of MOS for an admin without needing to go through the hassle of making people actually listen to the calls. That data could then be provided through analytics dashboards as an input into the overall health of the system.

Like A Rolling Stone

The problem with MOS is that it has always been subjective. Two identical calls may have different MOS scores based on the listener. Two radically different codecs could have similar MOS scores because of simple factors like tonality or speech isolation. Using a subjective ranking matrix to display empirical data is unwieldy at best. The only reason to use MOS as a yardstick is because everyone understands what MOS is.

Enter R-values. R-values take inputs from the same monitoring systems that produce MOS and rank those inputs on a scale of 1 – 100. Those scores can then be ranked with more precision to determine call quality and VoIP network health. A call in the 90s is a great call. If things dip in the 70s or the 60s, there are major issues to identify. R-values solve the problem of trying to bolt empirical data onto a subjective system.

Now that communications is becoming more and more focused on things like video, the need for analytics around them is becoming more pronounced. People want to track the same kinds of metrics – codec quality, packet loss, delay, and jitter. But there isn’t a unified score that can be presented in green, yellow, and red to let people know when things are hitting the fan.

It has been suggested that MOS be adapted to reference video in addition to audio. While the idea behind using a traditional yardstick like MOS sounds good on the surface, the reality is that video is a much more complicated thing that can’t be encompassed by a 50-year-old ranking method like MOS.

Video calls can look horrible and sound great. They can have horrible sound and be crystal clear from a picture perspective. There are many, many subjective pieces that can go into ranking a video call. Trying to shoehorn that into a simple scale of 5 values is doing a real disservice to video codec manufacturers, not to mention the network operators that try and keep things running smoothly for video users.

R-value seems to be a much better way to classify analytics for video. It’s much more nuanced and capable of offering insight into different aspects of call and picture quality. It can still provide a ranked score for threshold measuring, but that rank is much more likely to mean something important for each number as opposed to the absolute values present in MOS.


Tom’s Take

MOS is an old fashioned idea that tries valiantly to tie the telecom of old to the digital age. People who understood subjective quality tried to pair it with objective analytics in an effort to keep the old world and the new world matched. But even communications is starting to eclipse these bounds. Phone calls have given way to email, texting, and video chats. Two of those are asynchronous and require no network reliability beyond up or down. Video, and all the other real-time digital communications, needs to have the right metrics and analytics to provide good feedback about how to improve the experience for users. And whatever we end up calling that composite metric or ranked algorithmic score, it shouldn’t be called MOS. Time to let that term grow some moss in the retirement bin.

 

Fixing E-Rate – SIP

Embed from Getty Images

I was talking to my friend Joshua Williams (@JSW_EdTech) about our favorite discussion topic: E-Rate.  I’ve written about E-Rate’s slow death and how it needs to be modernized.  One of the things that Joshua mentioned to me is a recent speech from Commissioner Ajit Pai in front of the FCC.  The short, short version of this speech is that the esteemed commissioner doesn’t want to increase the pool of money paid from the Universal Service Fund (USF) into E-Rate.  Instead, he wants to do away with “wasteful” services like wireline telephones and web hosting.  Naturally, when I read this my reaction was a bit pointed.

Commissioner Pai has his heart in the right place.  His staff gave him some very good notes about his interviews with school officials.  But he’s missed the boat completely about the “waste” in the program and how to address it.  His idea of reforming the program won’t come close to fixing the problems inherent in the system.

Voices Carry

Let’s look at the phone portion for moment.  Commissioner Pai says that E-Rate spends $600 million per year on funding wireline telephone services.  That is a pretty big number.  He says that the money we sink into phone services should go to broadband connections instead.  Because the problems in schools aren’t decaying phone systems or lack of wireless or even old architecture.  It’s faster Internet.  Never mind that broadband circuits are part of the always-funded Priority One pool of money.  Or that getting the equipment required to turn up the circuit is part of Priority Two.  No, the way to fix the problem is to stop paying for phones.

Commissioner Pai obviously emails and texts the principals and receptionists at his children’s schools.  He must have instant messaging communications with them regularly. Who in their right mind would call a school?  Oh, right.  Think of all the reasons that you might want to call a school.  My child forget their sweater.  I’m picking them up early for a doctor’s appointment.  The list is virtually endless.  There are so many reasons to call a school.  Telling the school that you’re no longer paying for phone service is likely to get your yelled at.  Or run out of town on a rail.

What about newer phone technologies?  Services that might work better with those fast broadband connections that Commissioner Pai is suggesting are sorely needed?  What about SIP trunking?  It seems like a no-brainer to me.  Take some of the voice service money and earmark it for new broadband connections.  However, it can only be used for a faster broadband connection if the telephone service is converted to a SIP trunk.  That’s a brilliant idea that would redirect the funding where it’s needed.

Sure, it’s likely going to require an upgrade of phone gear to support SIP and VoIP in general.  Yes, some rural phone companies are going to be forced to upgrade their circuits to support SIP.  But given that the major telecom companies have already petitioned the FCC to do away with wireline copper services in favor of VoIP, it seems that the phone companies would be on board with this.  It fixes many of the problems while still preserving the need for voice communications to the schools.

This is a win for the E-Rate integrators that are being targeted by Commissioner Pai’s statement that it’s too difficult to fill out E-Rate paperwork.  Those same integrators will be needed to take legacy phone systems and drag them kicking and screaming into the modern era.  This kind of expertise is what E-Rate should be paying for.  It’s the kind of specialized knowledge that school IT departments shouldn’t need to have on staff.


Tom’s Take

I spent a large part of my career implementing voice systems for education.  Many times I wondered why we would hook up a state-of-the-art CallManager to a cluster of analog voice lines.  The answer was almost always about money.  SIP was expensive.  SIP required a faster circuit.  Analog was cheap.  It was available.  It was easy.

Now schools have to deal with the real possibility of losing funding for E-Rate voice service because one of the commissioners thinks that no one uses voice any more.  I say we should take the money he wants to save and reinvest it into modernizing phone systems for all E-Rate eligible schools.  Doing so would go a long way toward removing the increasing maintenance costs for legacy phone systems as well as retiring circuits that require constant attention.  That would increase the pool of available money in future funding years.  The answer isn’t to kill programs.  It’s to figure out why they cost so much and find ways to make them more efficient.  And if you don’t think that’s what’s needed Commissioner Pai, give me a call.  I still have a working phone.

Is It Time To Eliminate Long Distance?

Embed from Getty Images

“What’s your phone number?”

It seems like an innocuous question.  But what are you expecting?  Phone numbers in the US can vary in length greatly depending up on where you live.  I grew up in a small town.  My first telephone line was a party line.  Because there were four families on the same line, phone numbers didn’t mean much beyond getting you to the general location.  When we moved into town we finally got our own telephone line.  But the number was only four digits, like a PBX extension.  Since all phones in two had the same prefix, all calls were switched via the last four digits.  The day finally came when we all had to dial the prefix along with the four-digit number.  Now were were up to seven.

If you ask someone their phone number, you’re likely to get any one of several number combinations.  Seven digits, ten digits, or even eleven digits for those that do international business.  Computer systems can be coded to automatically fill in the area code for small stores that need contact information.  Other nationwide chains ask for the area code every time.  And those international business people always start their number with “+1”, which may not even be an option on the system.  How do we standardize?

Cracking The Code

Part of our standardization issues come from the area codes we’ve been using for sixty years.  Originally conceived as a way to regionalize telephone exchanges, area codes have become something of a quandary.  In larger cities, we use 10-digit phone dialing because of overlay area codes.  Rather than using one code for all the users in a given area, the dial plan has grown so large that more codes were needed to serve the population.  In order to insure these codes are used correctly you must dial all ten digits of the phone number.

In smaller locations still served by one area code, the need for 10-digit dialing is less clear. In my home area code of 405, I don’t need to dial ten digits to reach the Oklahoma City metro area.  If I want to dial outside of my area code, I need to use the long distance prefix.  However, there are some areas in the 405 area code that are not long distance but require dialing 405.  These are technically Inter-LATA Intrastate long distance calls.  And the confusion over the area codes comes down to the long distance question.

Going the Distance

The long distance system in America is the cause of all the area code confusion.  Users universally assume that they need to dial a 1 before any number to cross area codes.  That is true in places where a given area code covers all users.  But users also need to dial the long distance code to access users on different phone systems and in different towns.  It’s difficult to remember the rules.  And when you dial a 1 and it’s not needed, you get the reorder tone from your telco provider.

Now add mobile phones into the equation.  My friend from college still has the same mobile number he had ten years ago in this area code.  He lives in Seattle now.  If I want to call and talk to him, it’s a local call on my home phone.  If his next door neighbor wants to call him it will be a long distance call.  Many people still have their first mobile number even though they have moved to area codes across the country.

Mobile phone providers don’t care about long distance calls.  A call to a phone next to you is no different than a call to a phone in Alaska.  This reinforces the importance on 10-digit dialing.  I give my mobile number as ten digits all the time, unless I give the 11-digit E.164 globalized E.164 number.  It’s quick and easy and people in large areas are used to it.

It’s time to do the same for landline phones.  I think the utility for landlines would increase immensely if long distance was no longer an issue.  If you force all users to dial ten digits they won’t mind so long as the calls can be routed anywhere in any area code.  When you consider that most phone providers give users free long distance plans or even service for just a few cents, holding on to the idea of long distance calls makes little to no sense.


Tom’s Take

As a former voice engineer, long distance always gave me fits.  People wanted to track long distance calls to assign charges, even when they had hundreds of minutes of free long distance.  The need to enter a long distance access code rendered my Cisco Cius unusable.  I longed for the day that long distance was abolished.

Now, local phone companies see users evaporating before their very eyes.  No one uses their home phone any more.  I know I never answer mine, since most of the calls are from people I don’t want to talk to.  I think the last actual call I made was to my mother, which just happened to be long distance.

If telcos want users to use landlines, they should abolish the idea of long distance and make the system work like a mobile phone.  Calling my neighbor with a 212 area code would just require a 10-digit call.  No long distance.  No crazy rules.  Just a simple phone call.  People would start giving 10-digit numbers.  Billing would be simplified.  The world would be a better place.