Episode 19: Jason Jaworski On Vulnerabilities In AV & How To Succeed With Government Projects

this is a software defined survival where we explore how software defined systems are changing the business of IT today software defined survival it’s not just designing in ATV architecture or ninety architecture it’s actually working with the end user to go through an accreditation process to ensure that our two pieces that you’re connected to their network are not to introduce any vulnerabilities there’s very few manufacturers out there that actually take the time to provide a secure means to deploy their network long gone are the days of Hey we’re just appointing a standalone network if you’re going to get into the federal space in the market I you can’t just tell me a commercial integrated that those in and lives in the commercial retail space winning a government project and taking it works the same way and has the same processes is going to be in for a rude awakening hello there my name is Patrick Murray and welcome to software defined survival today’s guest is a veteran of the US marine corps and has over twenty years experience working in government and military systems on multi level classification videoconference systems command the control and operation centers ads really interesting stuff and he is currently V. P. of federal practice at unassailable solutions and this is a niche of the AV industry that I think is interesting for a lot of people so I’m really looking forward to this discussion with Jason your ski chase and welcome to the show a better things happen is there anything about that interrupt the introduction that you’d like to correct we’re done sounded us on on point so I like to start out by hearing about the origin story because AB is kind of this niche industry and people don’t normally grow up saying I want to be an AV so tell us how did you get started in a very yeah sure Hey I I think I have a kind of very interesting story and I I definitely grew up not saying I want to be a navy %HESITATION so interesting path about and I got the marine corps in nineteen ninety nine and round two thousand I was working for the defense threat reduction agency and they had a requirement to build a intelligence reaching into briefing center as well as a security operation center on the director the counter intelligence director came came down to me looks at the time I was the physical security engineer and said Hey can you build one of these on I said yeah sure should be should be possible and long story short I went to the evaluation of different control room solutions which included %HESITATION Barco in Jupiter and I found a company called active you are not and it was a software defined controlling solution and evaluated it and it met all the requirements that we had a so in two thousand I built to operation centers support the defense threat reduction agency and then I took a six year hiatus from the AV of industry I was in the counter intelligence he feels I was working as a skip accreditation officer and I was craving a skiff in a government facility and there was an active you a solution in that facility ana came under my purview to accredit that facility and the systems that were in it and so I was working with active you to make sure that their system was deployed in those environments for signal isolation and things like that and then they ended up making me a job offer and so in two thousand and six I handed in my government credentials and got into the control room an operation center market where with active you are standing up their federal practice here in northern Virginia and that that’s how I got into it like said the only from the street in counterintelligence field into the control room market and that’s really where I’ve been focused since then nice story the technology in general has a lot of acronyms but I think once you get into the military side of things it’s %HESITATION times one hundred so help us out here what’s what’s the skiff how sure skip is a sensitive compartmented information facility it’s a facility used to process intelligence information all right ends can you maybe give us a like a brief overview of what that first system you did with activity was like wait what what does a an operation center means you and somebody says build me an operation center what are kind of the core elements of that yeah well it can be a lot of different things and is very mission specific %HESITATION so there’s you know there’s never one one means to to solve every missions requirements are so for me in in two thousand that very first system how it was pretty simple the district operates the center really wanted to be able to display video from their CCTV system and at the time it was all composite analog video shows a bunch of composite capture cards built into a video processor for displaying those up onto a video wall so that the officers industry operates centers was a leadership to get quick situational awareness as do you our facilities within the national capital region of the Washington DC are so that was the main objective and goal of that a system and then the counter intelligence briefing the briefing center I had a very different role I really was for displaying dossier information on threats these two agents deploying overseas that supported the defense start reduction agency and to give them information about those threats and to do brief them on the way back out is to what threats they may have encountered why they were deployed overseas %HESITATION and gather that information so it was more about displaying information on individuals and on locations a Jew spatial locations from a mapping application standpoint once again it was fairly simple not too many windows going out same time so that was those are the first to operation centers right I have to overall with any operation center they though they all have very different missions in the objective usually is to gain situational awareness and in most operations centers nowadays are you the main component is gonna be a video wall some sort of but even that is starting to change to some degree so having your a large scale video wall to display information and so leadership can gain that situational awareness and now with the %HESITATION high resolution desktops you’re seeing more and more aware operation centers being deployed where the video wall is for your leadership to view from afar gain situation where it’s without impacting the operators on the watch for and the desktops are being used to support situation one is to the operators they can bring up you know six seven eight ten twelve videos on their on their own personal desktop monitors instead of having to you and you have a dedicated piece the video off for each and the mission that may be inside a an operations center that sounds a pretty fascinating a personal video wall at the resolution is getting back good where yeah you could kind of shrink all that down and still have enough detail for it to be useful so you mentioned situational awareness a lot I’m guessing video cameras things like that street cameras of maybe some TV feeds what what are some other things that might surprise us what what might be included what kind of sources might be included in there and what are the challenges on displaying them yet so of the cameras and in nowadays more more the camera feeds are coming you know to us we’re ingesting them over the network so he said to sixty four streams we offer those systems etcetera so those come from video management platforms are not coming necessary directly from the camera that you’re having to interface with you are interfacing in some cases with overhead assets such as drones are UAVs to bring an live feeds sport near the war for applications in other cases I even drone feeds us supporting our local law enforcement is is coming more more a standard are the other type of feeds on it would be like any I guess I say again because I I do so much but your applications that reside on a different PC with an operation center where that beat logistics whether that be our overall situational awareness type applications geospatial applications intelligence applications where that piece again are you man things that nature’s just static images being brought up into the video wall and in other yes source files being you played directly on the video wall verses of playing them on a PC and then your sourcing the PC to the video wall on it and more more you’re seeing where end users want D. of remote access to their PC so instead of having all your PC’s under your desk and in the military government application a lot of times there’s multiple classifications are so they can rack and stack all their PC’s in a data center in use a remote KVM functionality either through a network based solution or through a fiberoptic based solution like I think logical or something like restaurant in the annexe or a Matrox XTO things of that nature are you kind of more more of what platforms you’re seeing within the operations center so I’m you mentioned earlier the term signal isolation and I think this has a lot to do with what you’re talking about now we just described is easy to understand if the the rack room is is in that room next door but I imagine that’s not always the case it may even be in another building some time sometimes so what are some of the challenges of managing that on the network and making sure it is secure and %HESITATION doesn’t really touch any other networks that may not have that security sure what is the real challenges is that no two organizations %HESITATION menacing isolation the same way I sewed the method that you use for organization a may not be the same that use organization be even within a larger organization yeah you’d say you know the army or the marine corps the navy you would think if you went from one army installation to another army installation that they would follow the exact same guidance right and there are some very specific guidelines out there for separation of signals between classifications of what they call it red and black but not everyone implements and the same in every organization has the ability to use their own jurisdiction as to how they’re going to die there is a guidelines so in general there’s a people I use use the three foot rule for processing so PC’s anything with a processor in it need to review the separation between them on and then there’s a signal isolation on the cabling and that varies between what type of keeping your using and where it runs and how has to cross each other whether that’s fiber or whether that’s copper and you’ll find your should different organizations will use different methods my my safeguards when I’m designing a multi classification system is to always use fiber until the customer tells me that I don’t have to and then you will discuss the parameters of how that’s going to go about and bring the right people into the mix because early on with any government implementation do you wanna start talking about information assurance and system security I so those involved different people aware that the the the CIO or the Cisco or there is so I saw it is always the information systems security officer what the rules are going to abide by and and so those can be the RMF which is the risk management framework for how systems are yeah told from a security standpoint like the operating systems on different pieces and how they go about implementing RMS in in the different networks so that’s that’s all a big part of what we do it’s not just designing an ATV architecture or ninety architecture it’s actually working with the end user to go through an accreditation process to ensure that the parts and pieces that you’re connecting to their network are not to introduce any vulnerabilities and and that’s been a big part of where I’ve always felt the AV industry has fallen fallen down on others very few manufacturers out there that actually take the time to go through information assurance processes and to provide a secure means to deploy their their solutions onto a network because you’re long gone are the days of Hey we’re just applying a standalone network I’ve been deploying enterprise AV system since two thousand five two thousand six time frame and more and more that’s what the customers are going to it’s not the standalone so they want to make sure whether it’s a control system whether it’s a switch whether it’s a display on that are all connected to the network that when they run a scan on that network that they’re not gonna find any bone abilities are and if they do they want to know that they can be mitigated are so yeah that’s that’s part of what are the challenges that that I faced on a daily basis and I go back to manufacturers and say Hey we did a scan on your box right maybe it’s a Dante box and you know I go to a terror attack and say Hey you know I found that you’re using an old version of SSL or TLS yeah I need help and I will that’s nine are on a road map how do we get that done or I go to your view on I say Hey guys yeah we stand the software we found you know saying we found these list of vulnerabilities in how we go about maybe getting them and and I give those examples to the real world examples from a customer for command center that we’re working on right now where they need those %HESITATION bone abilities that were found to be that’s a really interesting perspective and %HESITATION I’ve heard this on this show before that most AV equipment really doesn’t meet the security requirements of %HESITATION many enterprise networks maybe a silly question because you listed a lot of them but what are some simple things that a manufacturer could do too well to help you out more yeah sure what I we still manufactures when I’m talking to about the government market is too hard in their box their their piece of equipment you prior to it going out to the market and and provide some documentation and as to how you went about Harding that piece of equipment and did you use a third party testing company to go through it versus yourself and can you show me that if you’re a switch that you have signal isolation if you’re if you have a network interface can you prove to me that when I run a scan I’m not gonna find in a bowl vulnerabilities or if I do do you have a secure configuration implementation guide that I can use because there are now some devices that are on the network I’ll use one for example the MDX I just did it and the X. deployment had like seventy NB exes honor and and they they have the ability to to be secure but you have to know how to set the to configure the unit so that they are using a secure settings where they’re using encryption on SSL and TLS are that their passwords are protected and the password to meet the guidelines that they’re getting updated that are part of active directory all these sort of things can really help me so I I encourage the manufactures the all the AB manufactures that have any sort of network interface to engage a third party I information assurance company and help them out locked down and harden their products in a before releasing them and then provide a secure configurations for the integrators to use are so that they have some level of assurance that when they deploy them on a customer’s network I’m not gonna introduce any bone abilities and once again with a set distance every single customer has a different set of guidelines or or interpretation to guidelines of how they’re going to implement security you’re never gonna have a fool proof means and I always tell manufactures is if you’re going to get into the federal space and then the federal market are you can’t just dip your tone it it’s your all in or nothing and it’s not a a once and done there’s no one time get get a approval and forever your product you meets all the requirements it’s a continuous updating because as we know there’s constant owner bill is coming out two for security technical security from a signal I intend to standpoint as well as in very much more so from the information assurance and over the network standpoint you know addressing viruses and malware and things that nature so I am I want manufacturers to to jump and stay and have a a a person at the manufacturer that knows how these church the great work and have that person be yeah the point person too in interface with that third party testing company to ensure that the there are deploying a a solution that’s going to meet the government’s needs thanks for that that was a really insightful ands yeah I like the way you didn’t put too fine a point on it that it’s it’s hard work if if you want to be involved in this space and I’d imagine the same thing would hold true for integrators as well that they’d have to really dedicate themselves to being a part of that industry that niche yeah very much so you know you’ll find that there are integrators out there that know the federal market and you can only know the lingo on they know what to look for they know how to in its you from a from an integrated standpoint they know the contract vehicles used to get partnered up to win projects but then they know your D. security protocols for it for just getting in a building a commercial integrated that does and and lives in the commercial retail space winning a government project and taking it works the same way on and and has the same processes is gonna be in for a rude awakening it does not work the same and it does take a specials a set of skills to operate as an integrator in the end the federal space and what you’ll find is in the the integrators who are successful in the federal marketplace a lot of them have previous military it as their employees are they came out of the government space that help them navigate in that in that federal market %HESITATION and and I think we we talk about before we started the recording is is having clearances right so you have to have your people on your staff to have a clearance you have to have the company itself cleared and it’s that’s a process in itself so yeah very much so integrators that have a federal group and that are focused on the federal market know the lingo know the requirements for information assurance TA compliance straight compliance on not used in many cases you’re not allowed to use the products are manufactured in China because of the threats and and because the trade agreements that are in place so you know those are things to know %HESITATION and and it just accessing those facilities what’s expected of you at those facilities insecure environments and knowing operationally a lot of times in the command center space we going to an operation center and start working to either update upgrade expand the TechTV technology that’s in that space arm and real world things start happening that require us to get out right then and there regardless of the clearance level that we have and you’re a an integrator that’s fairly focused is used to those sort of those disruptions is Hey mission comes first and when eight a real world scenario happens in that operation center get stood up and activated and things start happening you got leech and that you may leave for ten minutes you may leave for ten hours you merely for ten days on how long that is activated for until they can basic give you back here at the space to start working again us who you’re dealing with the the destruction that comes with with in the operations center space is also part of that and it’s it can be very frustrating I talk to you companies and some choose because of the frustration not to get into the federal space and I told you get it because either like said you’re in or you’re out you can’t just get your tone it interesting I really I’ve got two anecdotes about that I mean the example you gave was was pretty extreme that when something happens they need to use the rooms for a crisis situation but %HESITATION I remember I was on a project once and we needed an escort to work in the room we were working in and the first thing is you can’t take your phone in there so if one of your technicians of your smartest guy is working on that project he is offline you cannot get in touch with him for that day or days that he’s there and one of my colleagues went to the bathroom by himself one day and they didn’t like that at all that he just got up and took a pee on his own accord so %HESITATION yeah is very practical things that you need to be aware of and it’s a it makes a big difference on on on how you can approach these projects yeah I think that’s a great point did the just the simple fact is that a lot of people ask me why I carry a note pad there like Hey you know we all use iPads we all use tablets to do our work and like that’s great but in the space is not going to I can’t have my phone I can’t have a tab I can’t bring my PC and you know it’s it’s it’s an old fashioned tape measure in my my notepad and that’s how I that’s why I go to meetings because of the restrictions on secular technology or any in some cases you I would you guess watch I can’t even bring that in or a fit that all every all your electronics have to come off in that just goes with the I guess for me I’m so used to that have been doing it my entire career become second nature whereas a commercial person focused would probably you walk into an back home I got AI I thought I was gonna build a bring my laptop I thought I was going to be able to bring my phone I have these applications that I need to use a lot of times when we’re talking with our customers in a in a secure space yeah we start talking about those things early on is a are you going to provide us internet access inside your facility are do you have a laptop that we can use to connect to your network or multiple laptops if it’s a multiple networks to talk to you know for programming purposes or for configuration purposes know your DSP files your configuration of your encoders and decoders and control applications so yeah those are all good points that you it which makes it more difficult they’re they’re a lot more time consuming to implement he inside of a secure facility because you don’t have access to the people or whether there is kind of a it’s kind of a dark hole yeah interesting so on the one hand you’ve got all these limitations and restrictions put on you but on the other hand I’m sure that the latest technology is also what wants to be what they want to use and take advantage of so %HESITATION can you tell me a little bit about how maybe the role of software and networking has changed over over the years these kind of spaces yet one for me coming from active you a which is a software defined control room solution that’s where I got my start and they are a pure software solution they develop software and deploy it on me standard servers standard PC’s and they don’t label rand any of those so for me I’d and they’d have their own control software so much late in a U. tele Gee I thought active you was doing what you tell you did twenty years ago they’ve always had their own earn software defined control platform that ran on a PC that you can act from from an access the web browser that we just based on simple visual basic scripting also that that’s all I really ever knew prior to leading active you and coming into the the rest of the eighty market you for me a lot up there has been a whole lot this change I’ve always liked that model and I’ve tried to continue to use that model to use the the network to use the enterprise to integrate solutions into active directory are and to not be the standalone stove pipe solutions there are many more and a software defined control systems now I imagine you tell you because I’ve done a few deployments with them and I love their model it makes a lot more sense than happened that that black box that the customer has to call in it a programmer to support whenever they want to change versus Hey I can go into a web browser I can add a new device I can remove a device that can change devices I can change the look and feel of the user interface and I can do that on my own with my own ID guys that that I have on staff so that’s where I I do think things are starting to change %HESITATION more so within the industry as a whole on for me like said I’ve been doing that for a long time and so I’m glad to see that people are are catching up that’s interesting that’s so funny take so what it sounds like is on software defined systems for you just means more flexibility absolutely not being tied down to a single manufacturer and and not being tied down to an integrator either where I have to rely on them every time I need something changed so yeah I have a having that flexibility is huge and you might my first you tell your point was just that I had a customer who was an army customer and they said Hey look we have a restaurant system and had no issues with Crestron itself but they found it hard to obtain the funding they needed to make changes to their control system when they wanted it and they said Hey do you do you know something else that we could use that wouldn’t require any programming and I kind of scratch my head and I remember I I talked to a friend of mine give a shout out Shane Myers on he had told me about you Telek you said I could remember the name of the platform so I called the chance of Hayden was named a platform you’re telling me about the din requiring program and so he told me about you tell you so I gave Frank offer a call and he was very skeptical of me at first my my intentions I think he thought I was trying to get some information out of him but my intentions were true and we ended up deploying the first my first system within the government are with you tell G. and it’s still there today and it’s it’s working fine and so yeah it gave them a total flexibility they have to AV support technicians on site are they got trained up on you tell G. and after the deployment they continue to support it and maintain it interesting here Frank’s a great guy he’s been on the show before and a big advocate of software defined systems what I like about that story is that %HESITATION it goes back to this whole process isn’t systems and understanding how the federal space works they couldn’t get the funding to send a guy like me there to change the programming yep and and that’s where the softer so wasn’t really the technology itself was just really the the the you know the steps to getting a change done is is what was holding them back and was the argument for something software defined yet yeah exactly are you are you working on anything interesting enough now or in the future that you’d like to share with us I am working on things that are interesting own unfortunately because of the nature of the work we do on which is also a downside there’s not a lot of marketing I can I can put out there about what I’m working on a currently you know I I had a lot of times in federal contracts there’s a clause of of no marketing so you can’t necessarily go out and say I worked on your customer ABC’s your system and here’s what we did without getting permission from the end of the legal team at whatever organization that you’re working at so a lot of times any marketing is very generic in nature saying we did a DOD deployment in this general region of the United States or overseas and we use these general technologies because yeah they don’t they don’t want people to know exactly what technologies they’re using our wider using them and who they’re working with us so yeah unfortunately I’d I’d love this year I’ve done some really great projects and for me it that’s the reward is the frustration is there but the reward at the end is knowing that you’re actually help keeping people safe is huge your help and you know so I still have plenty of friends that are in the military employ your friends and our government other put themselves in harm’s way every day in knowing that you know I’m providing their leadership an operations center to help you know provide them better information to make better decisions are that hopefully will help keep them safe and keep him and protecting the assets of the United States is as awesome that that that’s a great thing is yell out I can’t tell people where the system that put him but knowing that they’re doing great things is is pretty rewarding excellent that’s great stuff I really think that a that is %HESITATION something that keeps a lot of people going is is seeing their systems in an operation and knowing that they they have a real world the fact it’s not just a meeting room that %HESITATION that collects dust somewhere it’s it’s something that is used and actually yeah in this case helps protect people and keep them safe so we’ve mentioned a few times in in this %HESITATION in this interview here how how complicated and difficult it is to navigate navigate this federal space where would somebody go or manufacture an integrator if they were interested in maybe just looking around and seeing if this is for them how would they go go about get starting with doing that where could they turn to for some help yes we offer an answer unassailable solutions we offer business consulting services to manufacturers and two other integrators that want to get into the federal space they know it’s alphabet soup they’re trying to figure out where all these acronyms are they want to get their products accredited where do they go to get a credit what accreditations they need how they go about that process word of the third party lab that they can go to to get their products tested in the Bible waited and who didn’t need to talk to the government to get their products approved for any true products list so yeah we’re we’re happy to help we’ve offered our services numerous times different manufacturers to get them started and where to go to get those services as well as where to find %HESITATION cleared people that want that can support the the federal projects %HESITATION and the class by space so it’s it’s not always just about you know understanding it but also needing and having the people the resources that can support customers want you do get into the market is there a if somebody would like to get in touch with you how would they go about doing that the best you know I’m a big guy linked in user I’m on there all the time so they can always looking up only ten you can also go to our our company website which is unassailable solutions dot com that’s it’s a company that’s run by a might my wife and I saw the you can look to see kind of what were our main focus is in when where where where we’re doing business excellent Chasin thank you so much for being on the show thanks Patrick if you or anyone on your staff ever considered themselves just in AV programmer join the club that’s how I used to feel I was just an AMX programmer or just Crestron program or whatever language of your choice is whatever it may be there’s generally this feeling in AV that we’re not capable of using modern programming languages and it simply isn’t true sure there’s a learning curve but once you get through it all other languages become easier to learn and it just expands the amount of options you have when designing a system it’s not an either or decision you don’t say I won’t be using these manufacture tools anymore it’s just you have a broader palate to choose from ends here’s what market day founder of idea box had to say about his experience with the online courses at learn AV programming dot com you know Patrick it’s funny how the smallest things can sometimes be the start of a really big ideas %HESITATION before I took the learn ATV programming dot com courses I was in that Terry I’m only a control system programmer kind of mindset rate %HESITATION when he came to new technologies or current technologies like Java script error or things like that for some reason I thought that was different from what I’m doing and what taking your courses flipped for me was not so much what I learned technically taking the courses it was the mindset of well wait a second I’m already doing ninety nine percent of what some of these most of modern programmers are dealing I just have to learn %HESITATION you know the other one percent and that’s really what I did so it’s really been kind of a big change after taking the course %HESITATION and I would really recommend this course to any integrator not only will obviously help their skill set but more importantly it might change their whole mindset %HESITATION which is more important and and and really show them new opportunities open the door so they kind of see problems through a different lens %HESITATION I gotta tell you one of the biggest changes for me was as soon as I come myself HTML CSS javascript and solve the you eyes that I can make with those technologies I just couldn’t sell a %HESITATION Crestron touch him again mark is a great example of somebody who takes new information and really applies it I know that mark still sells a lot of Crestron equipment but for him for his company for his customers for his business he needed a better you why he needed another option for user interface and modern programming allowed him to do that so the question is how can you use modern programming to improve your business please go to learn AV programming dot com and wherever you see a sign up button go ahead and sign up and you’ll get some free information to get a feel of my learning style and what kind of information is available and of course it would be an honor to have you in role in one of our courses and help you upgrade your skills and take this industry to the next level thanks for listening to solve find survival I hope you found it useful and maybe it inspires you to try out something new this week if you have any questions does software defined survival dot com and click the appropriate I’d love to answer questions on the air and if you’d like to help spread the word please subscribe comment and share it with your friends thanks