ÓÐÁϺÐ×ÓAPP

All right, we're ready to get started. The day is finally here. We're so excited, so good morning and welcome to DX23, ÓÐÁϺÐ×ÓAPP's inaugural Data Summit. We're so thrilled to host this in our newly renovated beautiful Innovation Center. This is a great opportunity to really exercise all the features that come with this center.Ìý

Ìý

We're joined here today by brilliant minds, brilliant minds from government, media, industry, and partners. Thank you all so much for taking the time to join us. For those I haven't met yet, I'm Judi Dotson. I'm the president of ÓÐÁϺÐ×ÓAPP's Global Defense Business and on behalf of the ÓÐÁϺÐ×ÓAPP team here today, thank you for joining us. We're here in the Helix, as I said; it's newly renovated. But one of the features of this Helix is it's highly connected to other offices around the world. So in addition to the audience here, we have over 700 people joining us spanning the entire globe, including Europe, all the way to the Pacific. We have about a for those on who are joining us uh through that connection in the room. We have over 100 people then and they also represent government leaders and as well as clients and our colleagues. So thank you all for being here.Ìý

Ìý

Before we jump in, I'd like to take a moment to tell those of you who are less familiar with ÓÐÁϺÐ×ÓAPP a little bit about us today. We're more than 30,000 people. We span the world and we support national security defense and commercial clients as well as civilian clients. We're proud that we work at the intersection of mission and technology and today's agenda reflects that what that means is that we take a mission-first approach. Our mission experts work side by side with our technologists to integrate the right technologies to help our clients achieve strategic innovation for the mission to do this. We utilize advanced technologies including AI NL. We integrate data. We call on our cybersecurity experts to be sure that it's secure and we use quantum computing to make a difference for our clients as leaders of the defense business.Ìý

Ìý

I am so proud that we're focused on supporting our US warfighters for all that they do for our nation and for our world. Now, working with Ki Lee. Ki, where are you? Ki's right here. Ki is the mastermind behind this event today and key. Thank you for all you've done to pull us together. But working with Ki and other leaders to prepare for today's event, made me reread the national data strategy for defense. And there's a quote that I think is particularly apropos for this morning and it's this every leader must treat data as a weapon system, stewarding data throughout its life cycle and ensuring it's made available to others. It's with that concept that today's agenda was formed. How do we seize the power of data as a war-fighting tool and enable those on the front lines, soldiers, sailors, airmen, guardians, and marines to use that to make the decisions and protect our freedom. So to kick off today's agenda, we'll hear from like our keynote speaker, Admiral James Stavridis.Ìý

Ìý

Thank you for joining us, sir. He's a retired US Navy officer and former supreme allied commander of NATO and he's the best-selling author of the book 2034 a novel of the Next World War. And I'm happy to say that there's a copy of that book for everybody in the room at the front. So please don't leave without grabbing your book following the keynote. We'll hear Q and A with Doctor Vint Cerf Google, vice president and a chief internet evangelist on the data evolution. And then we have two ÓÐÁϺÐ×ÓAPP leaders, Ki Lee and the T Springer, right to, to provide us a TED talk style uh discussion around data. After a brief break, we'll continue with an industry panel of distinguished speakers from Reveal A I data bricks, snowflake, and ÓÐÁϺÐ×ÓAPP and to close the event today. We are so grateful that we're joined by Ms. Margie Palmieri. She's the Deputy Chief, Digital and A I Officer for the Department of Defense. Great morning. So we have a great agenda. So let's get to it.Ìý

Um My name is Ki Lee. I am the defense uh sector uh technology officer. Um Wanted to welcome you all super excited about this. Um I am humbled to be um sharing the stage with our next guest. Um uh I was telling him, um I didn't want to run through his TV because I could be up here for 40 minutes. Um You could Google him if you don't know. Um But just like a Galactico football player, Pele Messi Marta, his name says it all. Um I wanted to introduce on stage, Doctor Vin Surf. You too. Good morning then. Um Thank you so much for joining us.Ìý

Ìý

Um I think uh Admiral ETSS did an amazing job in providing the context. Um But first of all, congratulations, 50 years um introduction of the, the internet and that disruptive idea for those of you who didn't know. Um I wanted to offer you uh any opening remarks. Well, actually, what I was gonna suggest for my opening remarks is we invite the admiral back that, I mean, that was possibly the most diverse. Um and you know, meaningful opening talk I have heard uh that in memory and the, the, the depth and points that he made so powerfully with the images that he chose, just shows you what a leader he is. Uh the ability to communicate so many different concepts of uh in such a short period of time and given his amazing leadership history, uh you're lucky to have gotten him to speak this morning. Um My reaction was I got to learn a lot more about this guy. Uh So thank you for inviting me to be here early enough to, to hear that opening talk. Uh Just a couple of thoughts uh Before we get to some uh interesting, I hope questions. First one is that you've all proven an important theorem. Number 208.Ìý

Ìý

The theorem means if you feed them, they'll come and uh here you are. So we prove that theorem one more time. The second thing um I want to draw to your attention is that connectivity is everything. And you heard that message in part from the admiral this morning. But with regard to the internet, it is so fundamental that I think we lose track of how critical it is. The fundamental design decision in the internet was that everything on the internet should be able to connect with everything else. And that, that has to be preserved. And what we're seeing is a fragmentation of that particular important tenet. Why is it so important? Well, the assumption was that every device that on that was gonna be on that internet needed to be able to communicate with every other device because you didn't know ahead of time what things would usefully talk to other things. If we elevate this to the worldwide web, by which I mean, the next layer of protocol, every web page needs to be able to refer to any other web page that gives it the power.Ìý

Ìý

And as soon as you start fragmenting and interfering with that rich connectivity, you start to lose the capacity of the system to produce value. Bob Metcalf, the inventor of Ethernet who uh just got the touring award this year, which is a very big deal. Um had a, an equation. He said that the value of the networks increased by uh uh a factor of N squared where is the number of participants in it? And that's, we actually don't agree on this. I think it's only N log N. But nonetheless, uh even if even that is incredibly powerful uh value. So as soon as you lose that particular escalation, then you lose the real power of the system and it may drain the capacity and utility of the internet faster than the gain was obtained when open connectivity was available.Ìý

Ìý

So I really worry a great deal about that and much of what I tend to do these days later, uh tomorrow, I'll be briefing uh the United Nations and the um global Digital Compact community on exactly this point. That connectivity is super critical, we have to hold on to it. There are many other things we want to hang on to and there are side effects associated with that connectivity. For example, the freedom to connect also grants the freedom to harm. And we should be concerned about that. We're very concerned about human rights, freedom to speak, freedom, to hear, freedom to share, freedom to find information, but freedom from harm is not listed in those universal human rights. It should be there. And that is something we seek. And you see countries trying to impose on the internet constraints in order to achieve this protection from harm. But you can go about it the wrong way. And if we go about it by losing connectivity, we will lose the value of that system. So I'll stop there. But that, that's sort of an important element in my mind today which I hope you'll repeat and think about, I appreciate, I appreciate that. Um I have a couple of questions um that we've uh drafted up but um wanted to offer up to the, the, the the audience here.Ìý

Ìý

If you have any questions, please feel free to, to think through them, we'll have some uh Q and A after afterwards as well. Um that you connectivity, I, I think that's a great, great point there. Um And when we think about today's kind of data patterns which are becoming more decentralized, which was introduced by the cost of the internet. Uh I'm curious to kind of understand your initial decentralized design pattern of the internet and specifically how you went about indexing that world wide web in terms of uh the uh the balance among the various actors in the system. Well, I still think this is a very distributed system and when you see concentration, sometimes there are good reasons for that. And in this particular case, what you see is an economy of scale, which is almost inescapable. I mean, if you think just a second, think about data centers and how they are operated, they are highly automated, certainly ours are. And I would assume this is true for the others. And that means that you can build larger and larger data centers and without necessarily having to scale the number of people to operate it, you take advantage of automation. Well, in that case, building a larger data center turns out to be much more cost effective than building a whole lot of little ones, we still distribute them for a whole bunch of other reasons, not the least of which is that you want reliability.Ìý

Ìý

And in order to achieve that, you have to make sure that the data centers are not subject to a common failure mode. We also replicate data across the data centers in order to avoid anyone losing it just because of the loss of of either a data center or access to a data center there. Should be no loss of access to information. So you distribute things, but you take advantage of economy of scale and that's exactly what you're seeing right now. But you're also seeing the pendulum swinging again. If you go back to um the 19 sixties and seventies, when Bob Kahn and I were first contemplating the internet design, the kinds of computing we had were large scale time shared machines. They were expensive. It made sense to try to share them among multiple people. And it made sense to connect them together. And then as the cost of computing came down, they got smaller and less expensive. So you got uh instead of the data center uh in the fish bowl, uh you had departmental computers that were the size of refrigerators and then they got smaller and you had desktops, then you had laptops, then you had pads, then you had mobiles. Um Now you have I O T devices with teeny weeny little computers that animate them. Uh And then you look at the technology and you say, oh, I can build the data center for all those teeny weeny little things and build this big thing again. Now we're starting to see the term edge computing showing up which is sticking computers between what used to be the edge and what used to be the, you know, the data center. And now there's a thing in between. And so we're just seeing computing going everywhere and doesn't that reinforce this importance of connectivity, everything needs to be connected in order for the utility all that computing power to be exercised. So uh my sense right now is that uh we're still seeing a mix of concentration because of economy and scale and distribution because of low cost. I love the pendulum anecdote. I I think um that's so apropo. Um And I agree 100%. Uh I'm curious um now we have data data everywhere. Um, from cloud to edge devices, we, we're collecting this data at an extraordinary rate. Um, you know, everyone's talking about A I, uh I'm, I'm curious to get your thoughts and, you know, I've had some initial conversations about data and related to A I A I is kind of the shiny pill right now. That's the nirvana state. Um, but your thoughts on kind of how we crawl walk of one and fuel that A I. So, uh, let me, first of all, thank you for the question. Second two points.Ìý

Ìý

The first one is I'm not an A I expert, I'm not a machine learning expert. This never stopped me from having an opinion, but, but I also want to, uh, make sure that you understand, I'm speaking for Vince search. I'm not speaking for Google in this response and the reason I tell you that is not because I'm trying to protect Google or something. It's just that, um, I'm not responsible for Google's development. Uh, and I don't want to speak for the company. Um Even though I um I work there, uh but I do want to talk about this, but I want you to understand this is Vince serves view, not necessarily uh Google's plans or views. So the first thing I would observe is that machine learning, let's use that word instead of A I because A I gives um the of intelligence to something which isn't exactly um what we're really seeing is the ability to assemble a huge amount of data that lets us simulate what human discourse looks like. When you look at the machine learning chat bots, what they have done is to AAA assemble and ingest a huge amount of examples of human discourse from all the text that they uh ingested. And they modeled what they have ingested using these very large scale multi layer neural networks. That particular mechanism allows it to react to textual uh inputs and generate outputs that look very intelligent, but they are only as intelligent as the content that was ingested. Now having said that I think you should also appreciate one other very important characteristic of these kinds of chatbots and machine learning tools. By the way, there are other kinds of machine learning which I think we should talk about. So don't let me leave off with only with the chat bot stuff. But in particular, even if you train the chatbots with factual information, they will get things wrong. Here's, here's the test. I tried, I went to, uh, one of the chatbots and I said, write me an obituary for Vince Serf. And you know, why would I do that? It sounds macabre right. But I did that because first I knew there was a lot of information out on the net about me. So presumably the chat bots will have ingested. That second obituaries are a common format and we do them every day. People die every day. Obituaries are written every day. So I assume that these bots would have ingested that format and would understand, understand that format. So it produced a credible obituary except that it gave me credit for things that I had not done and it gave other people credit for things I had done. It got things wrong. And why is that? Well, it's because of the way the text is generated. It's simply generating the, the next high probability piece of text that comes after the preamble that it's already generated. That's roughly what's going on a bit at a time. It's not quite words. It's, it's, uh, it's actually fragment uh of uh sentences and things like that, but it's just producing the next likely thing. And since you know what a salad shooter is, you know, there's the thing that you stick a vege vegetables in it like a carrot and chops it all up and it sprays the pieces into a bowl and then you toss it all together and then you walk, imagine that truth is a vegetable and, and you stick it into the salad shooter and you chop it all up into these little tokens and you spray them in the bowl and you mix them all up and then you serve the salad. Well, what you're getting is a word salad from the chat bots, which is filled with mixtures of truth and falsehood even when the sources are all true because it's conflating things and it shouldn't. So we should be super conscious of the weaknesses of the machine learning tools.Ìý

Ìý

Now, having said that I want to mention successes in machine learning which are jaw-dropping. Um, there is, uh, there's an organization uh in the UK called Deep Mind. It's one of the alphabet companies, a sister company to Google. And uh it had invested in trying to solve the problem of how proteins get folded up. You know, they get generated from DNA and they come out as a string, but it's a string of chemicals that eventually fold up because of the physics. And the question is how do they fold up and why do we care? Because it's the structure that determines how that molecule is going to interact with all other molecules in the body. And so knowing how things fold up, tell us a great deal about what that protein could do. And if you're looking for uh drugs to uh interfere with some particular physiological process, you need to understand what the shape of the proteins are. So they figured out how to do that by training the system to uh learn how to fold the molecules. And after it figured this out, and it got help from human beings who are literally playing a game of how does this fold up? They trained the machine, they figured out how 200 million proteins fold up, which now means that of the 200 million proteins that could be made with human DNA, we know how they all are shaped. So we have a clue about what they could do good and bad. So that's an example of machine learning that's really helpful. There are other examples like discovering asteroids that might be in an orbit that would cause us trouble that we don't need to and don't want to go like the dinosaurs did 65 million years ago. But in order to figure this out, you need to have a lot of data about where things are and what their orbits are and using machine learning to help you detect that is very powerful. So I'm a huge fan of using these tools, but I am also a believer in understanding their strengths and weaknesses and recognize risk. So do you know that the uh society of automotive engineers has uh uh analyzed self, not just self-driving cars but cars with assistive technology and they've created a uh a layered risk analysis of this intelligence in the cars. So the layer zero is that when the car has no intelligence and you're the person driving and it's all, all the safety is on you. But the next level up is the car has sensors and it can warn you that don't change lanes. Right now. There's somebody in your um in the, that you can't see in the mirror because that's the blind spot. And then as you work your way up to full autonomy, there's increasing amounts of risk that the system won't work the way it should. The same argument could be made for machine learning applications. We should be asking ourselves what are the risk factors in this application? What could possibly go wrong and what should we do in order to apply these systems in an intelligent way that minimizes risk? So my big fan and my, my big point here is that um we should be not abandoning these things, but we should use them smartly by recognizing what risks we are taking and then asking what can we do to mitigate those risks. So that's where I am on the A I and machine learning. I'm a big fan, but I believe we should be smart about how it gets applied. Can I um come back to you and tie the other some thoughts um distributed based on the internet and connectivity as well as your solid shooter analogy. When data is so distributed and you're creating salad with multiple different vegetables and potentially you have multiple different salad shooters. Your thoughts on how we access the data, how we verify the data, the truth of the data to your obituary analogy, how we assess the quality of that data? Um, let's get your thoughts on that. This is such a good question. You know, we run around saying data is everything, uh, and, and in some, uh, calculus it is, but it's the quality of the data that makes the big difference. And in fact, it's a high risk factor if you think about it because the more we depend on assembling data and using it in the way that we were just describing the more at risk we are to data being wrong may be deliberately wrong, maybe uh inter interfered with in some fashion in order to produce an outcome that we didn't want or intend. So the question is, how do we assess and keep track of the quality of the data? That's a damn good question. Prominence is the term that gets used a lot in the art world, you know, where did this painting come from? Do we have a history of it? Am I getting a fake or not? What about the data that I'm ingesting? Where does it come from? What evidence is there that it has value and is accurate? How do we do that? How, how do we keep track uh where data came from? And how do we qualify it? And I don't have a, a glib answer to that question. But your point, you're putting your finger exactly on the point which is that we should be very conscious that the data we want to rely on to train these systems has to be of good quality. It gets even more complicated when you think about it because over time, data can potentially become less and less valuable because it no longer is reflective of the current state of the world. So sometimes old data is misleading. Sometimes old data is super valuable. I mean think about the history of the earth that's captured in the layers of rocks. That's that's really valuable data, but it has its own provenance in some sense. So finding a way to identify the provenance and truthfulness or accuracy of the data is really valuable. That may mean in the scientific world, by the way, this is super important because you could easily come up with wrong theories based on bad data. So not only do you need to know about the quality of the data but you need to know how was it measured? Uh How did we calibrate the tools that uh did the measurement? Do I keep track of all that metadata? So that when I'm ingesting the data around a machine learning system, I keep that metadata around to help me determine the quality of the result all that we need to learn and and to use the Admiral's point, we're in early days right now of understanding exactly how to do this. And so in our excitement and zeal to use these new tools, we need to think as you just did about how to make it come out right by making sure that the data is real and accurate. So um when I think about data today, um I think about in terms of book ends on one book end, you have the prevalent pattern of bringing data to some centralized compute. We talked about edge computing. The emerging um pattern that we're seeing is bringing the process to the data based on your experience and you know, designing uh the internet. I'm curious to see if we can apply your lessons learned on how you actually indexed html pages in the worldwide web because that was actually relatively distributed. Um I want to see if there was any less learned from that, that we could apply to just data writ large both at the edge and actually centralized. Well, there are several things about where data is located that might be relevant. Um Privacy has become a major concern for everybody. And so keeping data local is very attractive, which is why being able to do some of these A I things or machine learning things locally is very helpful. Here's an example, we have an application for a mobile called Live Transcribe. It was developed by a deaf Russian who works at Google and uh he needed it because he needed some way of seeing the words that were being spoken. And his original design is data that was um uh trained uh on our cloud. And you had to get the audio from whoever was speaking from the mobile all the way to the cloud, do the transcription and then send it back as time has gone on. We've been able to move the artificial intelligence implementation closer and closer to the point where it could run in a mobile. And so it stays local and it never gets out there. So the privacy concern that gee what I just said said is sitting in somebody's cloud uh goes away. So there are things that we can do to um take advantage of this very distributed computing capability in order to control access to and use of the data. Um I just wanted to take the time to see if there are any questions. I think we might have time for one or two. I didn't want to take up the entire time asking the questions, any questions for uh vent from the audience, the microphone is coming. So, so we often talk about when you think of data and the analytics become information and knowledge is very linear. But given the power of A I and machine learning, can we also be thinking about the shaping of information resiliency by by going into a reverse? In other words, I want people to think this way. So how do I flood the system with enough data that drives the linear processing? I think that your point is should be extremely well taken because the social media have demonstrated exactly that. And we, we have discovered a remarkable leveraging capability uh by feeding certain kinds of information into the system. Uh We literally get a more than linear effect on populations. And this um that's actually kind of a scary thought that, that you've raised here. So the question is, how do we understand that? And how do we counter it? And I think that uh if, if I were, you know, to sort of start over again, not so much on the internet, but in this uses, uses of the internet. Uh I would do what Xerox Park did way back 50 years ago when they were first uh exploring Ethernet and the Alto Workstation. And the um the there was a, a um a text editor that uh was uh developed and a laser printer. These people, this is 1972 73. These people were living 20 years in the future and they knew that. And so they hired psychologists, anthropologists and sociologists to study how those 250 or so people interacted with each other in the con in the context of these technologies, we probably should have been doing that all along. And, and, you know, I think at Google too where we began and shouldn't, should now invest in understanding the dynamics of these kinds of computer facilitated interactions and, and the group dynamics of that. So, um so I'm thank you for bringing that up because it's an important point for us to keep in mind as we try to apply these technologies and avoid their negative consequences. That's really insightful. Thank you for that. And one more question. Yeah. So when you were working with the internet protocol, TT P, what were your thoughts about security in that? Um And for example, today, and you mentioned about, you know, we don't know where the data has come from. So should there be a data uh metadata, you know information protocol with the data? For example, when you use an M L model, you have a model card, you have all the metadata associated with the model. But when you get a data file, CBS, you have a raw data file, you don't know where it came from, there's no metadata with it. So how do you feel that in the context of should there be like a data protocol added to data? And what did you think about when you were developing the internet with TCP and security? How did that? This, this is uh like the final exam, it says uh de describe, describe the universe in uh 25 words or less, give three examples, right? OK. Let me parse this a little bit. Uh First of all. Uh let me take the data side first, I think the idea of capturing the metadata about where the data came from is super important. And in some sense, we should start saying I am not interested in your data unless the metadata comes with it. Where did it come from? How is it generated, how you know how is it produced, how has been filtered? So uh on that score, I think we should become increasingly insistent on having more knowledge about where the data has come from. Its, its uh quality uh with regard to security. I, I'm often asked that question and I have to tell you that starting in 1975 I was um flying out here from uh Stanford to NSA to work on a secure version of the internet. Um The problem we had is that the a lot of that initial security, this is before public key crypto was available was classified. And so I couldn't share a lot of what we were doing with the other graduate students who uh were working on the internet. So there was a, a kind of a parallel uh work going on. Some of the TCP design took advantage of the, our understanding of what uh crypto requirements it would have to um address, you know, in the, in the structure of the protocol. Um by the 1978 I'm here and I'm running the program at ARPA and um I'm ready to freeze the protocols. Two years before in 1976 the first paper on public key cryptography is published by Whit Diffie and Marty Helman at Stanford called on new directions in cryptography. They didn't have any algorithms in that paper. They just said, if you found um you know, um uh functions that have these properties, then you could do these really cool things. And then a couple of years later, we get the R S A protocols, but I'm trying to freeze the TCP/IP protocols so I can build them on as many operating systems as possible in order to test the system. And so I came to the conclusion that I could retrofit a lot of the public key stuff into the internet. But I didn't want to do that in advance of getting the basic protocols demonstrated. So we focused our attention from 78 to 82 on getting them implemented on as many operating systems as possible. While this new form of crypto was uh was evolving as opposed to the kind that I had worked with in the mid seventies, which involved key distribution centers that were, you know, clunky by comparison. So um I defend myself um by arguing that, that we could, could and have retrofitted a great deal of the public key crypto into the system. We still need to do a lot more of it. R P K I is needed. We need to do DNS sack. We, we've got quick protocols and T L s and so on. But we need more of that.Ìý

Ìý

But I have to tell you I was looking back and thinking, OK, I'm gonna put crypto into this thing as quickly as I can. I'm thinking graduate students are they the people you'd want to rely on them for key management, you know, and the answer is maybe not, they're being distracted by final exams and getting the dissertation done. And so I didn't feel strongly that I needed to jam that into the system and impede its ability to demonstrate capacity. So, so there, that's my answer that um I wanna wrap this up with one final question for you. Then um you previously said in an interview, I love to build things that connect people. I favor long term projects where the technology needs to catch up with the aspiration. I'm curious, what would you say is today's aspiration where the technology needs to catch up? Wow, this is interesting. So uh two book Analysts to borrow your earlier um analogy. Uh 1998 some of us started working on an interplanetary internet design. Uh wondering uh thinking that we were gonna need it 25 years later here. It is 25 years later and we need it. We are on our way back to the moon commercialization is happening. So, trying to look at today and asking what do I need to have 25 years from now? In addition to having that uh interplanetary internet design done and working, uh I think that the biggest opportunities right now have to do with computational X for almost any value of X that you can think of whether it's linguistics or biology or physics or uh economics. And so on computing has become and it will be deeply embedded in our society and our ability to use those computational methods and recognize what the risk factors are when we apply them is I would say very immature at this point. And so thinking about the introduction of computational methods in virtually everything that we do is the need that we have now that we have to have satisfied and understood and implemented, including whatever they. Well, let's say the constraint frameworks are so that we use those technologies wisely and safely. We need to understand that better. That would be my, my highest priority right now is what are the risk factors of a society that's so heavily dependent on computation and data. And what should we be doing in order to make that work well for us as opposed to becoming a potential hazard. So that's where I would come out on that one. But, well, Vin uh I appreciate your time. I think we could talk all day. Um We've had several conversations. I had the pleasure of uh picking your brain. Um Thank you for your time, your insights. Um And your legacy, I think you have been a massive disruptor. So, thank you. Thank you. Wow. So um we started with uh Admiral ETSS talking about the, the mission context and framing it. Just had a conversation with uh one of the fathers of the internet. Um I think you all should feel a little sorry for me because I have to follow that.Ìý

So with that, I'm actually going to call up the real deal matisse to talk through how this actually works from war fighter perspective matisse. Actually, I don't need that. Thank you. Thank you, sir. Well, good morning everybody. Uh Wow, this is an impressive crowd. Good looking too. Uh But thank you Key for that introduction. And uh I am excited to be here in the Helix because for so long, we've been uh trying to get this fully operational and here we are. So this is absolutely beautiful. Um We've had some fantastic discussions this morning and, and I just wanna start off by saying that I am no, Vin sir. It's a pleasure to meet you, sir.Ìý

Ìý

I remember studying you a few, a few decades ago and it's a pleasure to see you in person. Um But I am someone that's used data uh throughout some of my operational time flying in the Navy. And so, um it's just been a fantastic discussion this morning uh on data and how we especially as a systems innovator, uh an innovator, hopefully of choice uh can leverage data to help the war fighter be competitive on the battlefield. So as key said, my name is Matisse Wright Springer. I think my boss, his boss's boss got my name wrong this morning. Uh But Matisse Wright Springer and I graduated from the Naval Academy back in 1988. So a long time ago. So key. Thank you for saying things like tower records and AOL because you made me feel like, ok, maybe I'm not as old as I'm feeling. But in 1986 I was a second class midshipman at the United States Naval Academy. And in 1986 the movie Top Gun came out, you know, all that good stuff, right?Ìý

Ìý

And it, it was in my junior year in college. So that's about the time that midshipmen are really beginning to think about what are they going to do once they hit the fleet. So I tell you after seeing that movie and I probably went to see the movie with a bunch of my classmates from the Naval Academy. So we all get back to Bancroft Hall and we're all walking around saying, hey goose, hey Maverick, you know, and we're all thinking we're gonna fly. Uh So I had the pleasure after graduating of uh of attending the uh the Navy's flight school. And in 1989 I earned my wings as a naval flight officer and became the nation's first black female naval flight officer. Thank you. Bye. So, so one of the great things about the career that I've had so far was flying in the Navy and I flew in an aircraft called Takio or the E six. Uh, the aircraft is amazing. I had more than 2000 hours airborne in the aircraft and I'm gonna tell you a couple of sea stories about it later if time permits. Um, but a little bit more about me because I think it's important for you guys to know a little bit more about just uh about me, not just the naval experience, but I've also had the pleasure of serving in the government a couple of times as well. First, as a presidential appointment as a White House fellow.Ìý

Ìý

And second, I served as a senior executive at the Pentagon and I ran the Secretary of Defense's Office of Industrial and Manufacturing Based Policy, which was just a phenomenal opportunity set. It's a small office there in the Pentagon. But man, it has a mighty punch. And again, it looks at the entire defense industrial base to ensure that the defense industry members like booz Allen uh are able to provide for the war fighter no matter what the challenge is. So outside of government, I had the opportunity to, to work in a number of different companies, big companies, uh five big companies. As a matter of fact, I've, I've worked for a company called S R A that got acquired by general dynamics. I've worked for soul helicopters for Sikorsky Aircraft, which is now part of Lockheed Martin and of course, I've been here at ÓÐÁϺÐ×ÓAPP and actually, this is my second time here at ÓÐÁϺÐ×ÓAPP because I'm having so much fun. Uh and I came back to ÓÐÁϺÐ×ÓAPP as a senior vice president in 2021. So today I am overseeing a part of what's called the aerospace account. And my customer is actually the Air Force. I told you all about the Navy and Top Gun and all that. But my customer today is the Air Force and uh we have some other dod uh customers, but primarily I'm focused on the Air Force.Ìý

Ìý

And as a senior vice president, I have profit and loss responsibility for a sub account called Enterprise and Nuclear Solutions. So the takio aircraft is based on a nuclear concept. And so I'll get into more detail with that, but I work with a fantastic group of people in E N S and I tell you they're just phenomenal. They are software developers, they're software engineers, they're program managers across the board and many, many, many of them are way smarter than me and make sure I'm gonna make sure they're up here on the stage to talk about data. But as it stands today, I wanted to kinda share a little bit more about my background to set the stage for the example that I'm gonna share with you guys in a little bit. So again, my team on a daily basis that we are constantly challenged with some of the toughest problems and toughest challenges that face the Air Force and it's always around data of some sort. So we heard from Admiral Stavridis and we heard him talk about the geopolitical spectrum, right? Everything from China to Russia, Ukraine, et cetera. Um All of these entities are introducing new concepts and new things to our war fighters. And we have got to give our war fighters a competitive advantage on the battlefield. If we don't do that, we don't deserve to be here.Ìý

Ìý

And so I think of, of uh serving my air force and working my, with my team today truly is a privilege because it is a privilege for me to help use the talent of my team and the, the experiences that I've acquired over the years to help the Air Force with their toughest problems. So we just heard key and, and um Vin talk about seizing the power of data uh and using time as a weapon and every day our Air force needs help with just doing that with all the data that they have to manage. Um We often hear if you, if you're following the Air Force, you often hear the chief of staff of the Air for Force talk about accelerate change or lose. And one of the things he's now saying, so he was saying that in the very beginning of his tenure. Now, one of the things that he's saying is accelerate change and win. And that's the phrase that I want to just kind of capitalize on because it's all about winning. It's all about giving our war fighters the ability to win no matter what the challenge is that either the Chinese, the Russians, a counterterrorist entity, whatever um uh comes in front of them. And I think we can do it with doing kind of three simple things. First, we have got to help simplify the data that they're using. If we can ensure their data is simple and easy for them to understand, they have a better chance of being successful, no matter what the mission is. The second thing I want to make sure we're doing is getting the data to them when they need it.Ìý

Ìý

If we can get the right data to them, nine times out of 10, they're gonna knock it out of the park. And the third thing I think is really important is we have to adapt the data, like putting it in, in the right format, putting it on the right tool. We have to adapt the data for the mission that they're pursuing at the moment. So when we do those three things, I know for sure we're gonna give the, the uh war fighter the competitive advantage on the battlefield because I've been sitting in that seat, right? And that's what I needed from industry when I was out there. It's all about the mission and speaking of the mission. I see this beautiful airplane behind me, isn't it just beautiful? I mean, it is clean lines. It's got uh you know, good balance, good symmetry. Uh And, and again, it's uh it's an E six aircraft and it's also called Takio or, you know, we used to call it the Takao bird, the Takio stands for, take charge and move out. So I'm gonna use that theme throughout the remainder of my talk this morning. And you know, the thing that I think is most important is the Takao mission. We each of us have a responsibility to take charge and move out. And I believe this airplane behind me is the most important aircraft in the entire defense industrial base. Actually, it's the most important aircraft in the world to include what's in the commercial sector. So I see Kenny Smith over there who's a fighter guy and he's probably gonna tackle me once, once I get off this podium for saying that, but, and he's probably shrugging his shoulders like no, no, no. That thing ain't nowhere near as good as my fighters.Ìý

Ìý

But the reason I think it is the most important aircraft in the entire defense aviation portfolio is because it provides a connection or communication capability for the most senior leaders in our government to connect with military operators around the world. You can't say that Kenny about a DAG on F-16, they can't do that. This thing has some compute power and some sensors that are able to collect data, do some things with it and then transmit it unlike any other aircraft in the United States of America or in the world. And for that, because it is known as the doomsday aircraft. And I hate saying that because it sounds so morbid but you know, some people call it the doomsday aircraft because when that airplane is flying its mission for real, some really bad stuff has happened in the world. It could be a nuclear disaster somewhere. It could be some huge climatic disruptive thing that's happened in a part of the world that's essential to our nation's security. Or it could just be some major malfunction of our normal communications capability. Be it inadvertently interrupted or intentionally interrupted that this air aircraft can provide continuity of operations for our war fighters and our most senior leaders in government. So that's why Kenny I think is the most important aircraft in the entire portfolio. All right. So, um what we have to do is we have to help ensure that the operators that are performing this kind of mission are successful because if it's a nuclear or catastrophic activity or event that's happening, we have got to figure out how to get them alert, airborne and capable of doing their mission. So let me, let me just talk a little bit more about the Tao bird, take charge and move out again. We have got to help manage the data that our war fighters are leveraging. We got to do those three things that I talked to you about. We got to keep the data simple. We got to get the data to them when they need it. And we've got to adapt the data to the mission that they're performing, not expect them to do it. And I'm gonna try and give an example of real time. Uh this aircraft. Um one of the missions that I flew a few decades ago. So one of the cool things about this aircraft again, it's the E six tac, it can refuel, it can refuel while airborne. And most of the time you get the small fighter jets that are refueled and taking on fuel so that they can go and do some kind of fighting mission. Right? But with this aircraft, it is really, um, an amazing thing to actually be in the cockpit when you're about to refuel. And let me, I'm gonna, I'm gonna shift gears a little bit and talk about the cognitive load that is placed on the operator during a mission like airborne refueling because you know, when you're on the ground and you have to refill everything's easy. You stay out on the flight line, the tanker comes over, they put the hose in, they refill the aircraft and good to go like there's not a whole lot of danger with that. But when you've got 100 and 50 tons which is basically what this aircraft is about £340,000 and it's tanking with another aircraft about that same size. So you got these two big pieces of metal coming together in an airspace where everything you're taught, if you're trained in the E six and the Takio mission, everything you're taught is to steer clear of other things that are in the air. But during this air refueling part, you're expected to get very close to a large aircraft. So you're at altitude, you're, you're feeling that heartbeat come up a little bit, right? That cocker factor of human beings is escalated.Ìý

Ìý

Uh So you've got, you're in the cockpit, you put, there's this thing that comes up right where that little arrow is, I don't know if you can see it from your seats, but, but there's a probe that sticks up out of the aircraft right there above where the pilots are sitting as a navigator. I'd be sitting right behind the pilots and behind me on the other side or, or adjacent to me is the flight engineer. So, so first thing you see is that, uh you see the pilots kind of, you know, you have a checklist, so you're calling in response to the checklist. So the pilots put that probe up and when that probe gets put up, it changes the configuration of the aircraft. So then you hear some, some turbulence. So your, your, your headset has to be a little snugger around your ear so you can hear the communications on the radios. So the sounds in the cockpit change, the refueling probe gets extended. The communications over the radios is even greater than at any time during the mission because you're about to get 100 and 50 tons right next to another 100 and 50 tons in the air. You're 30,000 ft in the sky or many, many thousands of feet above the ground, you're flying at hundreds of MPH each aircraft, right? Hundreds of MPH. And there's turbulence. So you can't quite control if it's a smooth flight or not. Right. So, the first thing you see is you see the boom on the tanker in front of you because the aircraft kind of comes up beside you. Uh, if you're, if you're in the refueling area comes up beside you, it drops the boom and it looks like that dag on thing is about this big. But then as the air air, uh, as the distance creases between the aircraft, man, that thing looks huge, you see the basket and it looks like it takes up the entire windshield of the aircraft huge. So again, that pucker factor goes up, your heart rate rate goes up, it feels like somebody just turned the temperature up in the cockpit about 100 degrees and you're like sweating, what the heck is going on? You're hearing things differently because that probe is up, you might smell a little bit of the gas as it's coming into the aircraft, but everything is just escalated. So the cognitive load of that war fighter is enhanced significantly than just level, smooth flight, like on an air on a seven oh seven that you might be taking from here to California, everything changes. Right. And so one of the things that we have to do when that cognitive load of that human in that cockpit is increased, it's no time for them to try and figure out. Ok, what's this data mean over here? You've got to simplify that data, you've got to get them the right data on time and you've got to understand the mission that they're performing. If you don't, we're a step behind and we don't deserve to be in this business. So let me give you real quick. Um an example. So one time I was out doing an air refiling mission. So I'm in the navigation uh station seat and the, the way I always like to work with the tankers because it is very, very dangerous, right? And, and I don't want want to be responsible for creating some mid air collision, right? So I always like to get to the tanking point as soon as possible. And for those that have never air refilled, one of the things that's really kind of interesting about tanking is no matter where you tank, it's never where you've tanked before. So it's always a new place and think of it as an invisible big oval pattern in the sky. And so what's supposed to happen is the air traffic controller is supposed to keep all other aircraft out of that area because these two big large hunks of metal are about to come together very soon. And so you don't want any other aircraft in there. So one time, uh we're, we're flying an airborne mission, we're in the pattern. I've got my aircraft in the pattern, got the headset on. So I'm talking to the air force tanker that is scheduled to meet me at a certain point, certain altitude, certain time. All that is checking. Good. Good, good. I look on my radar. Everything's good. I can see the, the, the, the tanker and my desire was always to have the tanker come off of like the four o'clock position. All of a sudden I look at my radar and around 10 o'clock I see beep and at back then the data would come and then it would disappear. So I'm talking to the, the tanker that I'm expecting at 40 at my four o'clock position. Good to go. I see that also in my radar beep and then it goes away and I'm good, I'm talking chatter, chatter, chatter. Then all of a sudden beep and I'm like, wait a minute, that point is getting closer to my refueling pattern. That shouldn't be happening. Is what I have to say. And then I say, all right, they must be in the wrong place. They'll know to turn all of a sudden, no beeps at all. So I'm like, check good. They've turned then about 20 seconds later. Now again, you're going 100 miles an hour, decreasing, bearing, decreasing speed coming at each other. So all of a sudden the beep and, and now the, that second aircraft which shouldn't be there is in my pattern, not where it's supposed to be. And I'm like, whoa what happened? So then it goes away and again, the in intermittent receiver that picks that thing up. Now I gotta wait another 8 to 10 seconds before it comes back around. I don't have time for that. Right? Cause we got to figure out what the heck's going on. Air traffic control. This is aircraft blah, blah, blah. There there appears to be a second tanker in the area, please clear. So get the and, and that all has to be done while I'm, I'm talking to the right tanker, right? I'm making sure they're on course so that we line up, right. I'm also talking to my pilots. I'm also sensing all those things like I said, who just turned the heat up in here to 100 degrees at this point is what it feels like. Right. So sweats coming down my face and all of a sudden the next iteration on the radar is a beep and I see it turning away from the pattern and I'm like, good. Oh, what the heck was that thing doing here? And so the, the, the reason I wanted to share that example with you guys is because at that moment when I recognized there was a second aircraft unscheduled tanker and it was be lining, it knew it was behind schedule because it needed to be at a certain point closer to, to me. So it was, he had pushed the throttles up and was trying to get there quickly, was changing altitudes I get on the line with or on the radios with the, with the air traffic controller saying, hey tell that aircraft to increase altitude, change direction, slow down, get away from my pattern. So all these things are happening like very within seconds and you've got to manage that. So as an operator, when I'm working with integrators like us here at ÓÐÁϺÐ×ÓAPP, I need you and all of us to understand what am I having to manage? What is the cognitive load of each member of that cockpit? Now, my pilots are just focused on staying on the course that they're supposed to fly, right. And so they're working on that. My flight engineer is looking at the fuel gauge and making sure that once we're connected to that tanker and yet this other one is still coming that, that the, the fuel that's coming on board is registering appropriately. And things are on track with that. Now, I've got to make sure this other aircraft clears the area because we're about to get at the, the, the 12 o'clock position in the pattern. And boy, he's over at 10 o'clock, he needs to get the heck out of here. Right. So, it's those kinds of missions where we have got to understand. What is that operator experiencing? What is their mission? Have I kept the data that they have to see clean enough and with that intermittent pulse on the radar, it was not clean enough, it was causing me distress. It made me, you have to do something that was not in the checklist, that was not a part of my plan. So while I'm talking to an air traffic controller to get that aircraft out of the area, I'm kind of overwhelmed and not able to try and figure out how come that dag on radar is every 10 seconds or 20 seconds. That's what we should be doing, right. So, so I share that because if the cognitive load had been appropriate, I would have never had to do what I had to do in that example. Um And so we have some everyday things that we're experiencing to help. In my case, my team E N s with the Air Force problems when we think about. No kidding. The real world implications of what we're doing to help them manage their data. We have got to take charge and move out. So thank you all for listening. Yeah, so, so again, you're right, sir. So, so Mr Surf just asked, what the hell was that airplane saying as a, as a naval aviator, I'm telling you, we were saying a whole lot more salty things than that, like, you know, especially my flight engineer like, you know, I, I'll keep it clean here. Uh But, but yeah, it was, it was incorrectly scheduled to also tank with, with the navy. Now back then in the eighties, remember we were trying to become more joint and so the air force and navy working together was getting more sophisticated, but it's nowhere near the way it is today, right? And so one of the things I believed was, first of all, I had no idea what radio frequency that second tanker was on or I would have called him my damn self and said, get the head out of here. I'm already tanking with air force tanker. 123. I don't need air force tanker 456 to clear the clear the area, but I had no idea what radio frequency they were on. Right. So I had to then find an air traffic controller who knew all the airplanes in the air at that moment and could connect with that tanker and say aboard that mission, there's already a tanker there and explain to them. But you're right, Vin, I mean, it was like what is that second aircraft doing there? This is a perfect example of information missing when it was needed. And we should have understood that when you design these systems. This is a fantastic example of a moment when you needed a piece of information, what's the frequency of that aircraft and you didn't have it? So we should be thinking not only about the cognitive load and the sweat and everything else, but we should be also thinking about the system design and the complexity of having, you know, integrating these things together. What were we missing and what should we have done and what should we do now? So that that doesn't happen. So that's the second point out of your example. Yeah. And so you're right, you're spot on and uh the the beautiful thing is it would be reported in open source material. If there were some incident that caused a midair, somebody in here would see it in defense news or breaking defense or whatever and, and that's never happened.Ìý

Ìý

So what I believe again, this was happening in the early nineties time period. I believe that they have done some things, we the systems integrators have done some things to make sure that kind of problem doesn't happen again. And remember when I first got to the squadron, I was actually flying C 1 30 out of Hawaii. This aircraft was delivered to my squadron in Hawaii about six months after I got there So it was new to the Navy arsenal as well. So there were some navy things that we were learning for the first time with how to interact with the Air Force. Now, the, the uh the squadron is co located at Tinker Air Force Base, a navy squadron co located there at Tinker Air Force space and things are a whole lot smoother. But I say all this because the ultimate thing that we have to embrace, to ensure that our war fighters win regardless of their mission is we've got to embrace change. This airplane has changed, it has way more sensors on it. Now, in 2023 than when I was flying in 1989 to 1992 the the the crew that's on board is trained differently, right? So how do we ensure we're keeping ahead of their mission? So that instances like that do not occur? Like how do we ensure that happens? That should be our challenge every day. That's the thing that wakes me up early in the morning and helps me stay up late at night to solve those kinds of problems. Again, today, my customers, the Air Force, but tomorrow it could be back to the Navy or to the army or whoever, right. So I appreciate your, your comments, sir and pointing out that that was a great example. I'll take a bow for that. And again, thank you guys for uh for giving me a part of your morning?Ìý

Um I wanted to conclude today um with a, our clean up and the heavy hitters. Um I wanted to introduce Mimi on stage and uh Miss Margie Palmieri, Deputy director CD A O. Well, thank you, everybody. And uh thanks to, to booz Allen for having me. I'm currently uh guest hosting uh C Span's Washington Journal, formerly uh hosted Government Matters on uh ABC seven. And uh just really excited to be here and to be with you, Marjorie, uh I, I don't need to introduce you the Deputy Director of the Chief Data and A I office at the DOD. So thank you for, for doing this and, and this guy is gonna join us too. It's a little bit intimidating. But before we jump into today's challenges, uh give us an idea of how dod s approach uh to data has evolved over time. Um Well, in the beginning, uh when the dod was created uh no data. Um So, so data is an interesting uh question for our focus area because the Dod didn't start with data. Uh I think uh I'll, I'll go back to the net centric days where we started with networks and we always knew that we wanted to connect different capabilities together. Um But we first looked at it through the networking type of question, how do we connect everything else uh via tactical data links or satellites or uh or the internet? And I know you guys have uh sort of earlier today. I got to hear him talk, uh which was fantastic. Um But over time, we realized that um the networks were really just the path for accessing data. And even if you could get data through the network, what arrived, was not necessarily consumable in the exact way you wanted or maybe it wasn't exactly what you needed. Um And so the focus on data has really shifted. Um I don't know the exact timing of that. Um But I would say in my own experience in Dod, which uh is, is a couple of years old um have really watched us go from kind of this integration of systems via networks to this um software based or API based approach to data access. Um You no longer have to all uh be in a common data hub. Um And what we're thinking about the CD A O is how do you embrace this decentralized data architecture, which is very much uh consistent with D DS culture. Um D U D is a huge organization. We are one of the, the largest, if not the largest in the world uh with all of our various components globally distributed across the uh across the earth. Uh very diverse uh when it comes to different domains, land under sea, air cyber uh surface um and, and space. Um and each one of those communities and those functionalities has a different approach and a different expertise associated with it. We love to say, you know, bureaucracies uh are, are tough but they're specialized. Um I really love the fact that there is someone who cares about the water of a submarine and they do not care about how that relates to the National Defense Strategy. There are layers above them that will make that connection, but that expert is, is a skilled expert in what they do and it is necessary for the department to have. Um So how do we embrace this idea of specialization but also recognize that there are these integrating components that have to come together. And so I think we've gone from kind of a network based way of integration to now a database way of integration. Um And I think that's really consistent with our culture and something uh exciting. So what's the top data challenge at the G O G? Yeah. Um Our director Craig Martel uh who came in from uh Silicon Valley, he was the director of A I uh for for Lift and dropbox and linkedin um really thought coming in, he tells a story all the time. I'll steal it from him that um convincing people that data was important, was going to be one of the main challenges that he had coming into Dod. And he said that's absolutely not the case. Everyone in dod understands the necessity of data, the power of data. Um The next challenge I think is to give that expertise, uh not expertise, but get that understanding over to uh leaders around how they need to effectively manage data. And I I personally think the biggest challenge is this concept of data as a product. And so in dod for uh as long as anyone can remember, we've acquired systems, our acquisition processes were designed to acquire systems, we deliver the system, it is tested and validated and then um it goes into sustainment. There's no uh very rarely is there a data uh product focus for that system. So I I came from the navy um a combat system and, and the radar spy radar on a ship uh were delivered as a package and they talked to get to the spy radar, talked to the combat system which um was then kind of inside of the ship's um you know, the skin of a ship. And, and that was that relationship. Uh over time, we realized that that radar data was really powerful and that combat system was actually creating new data as it fused it with other sensors and and uh things in the environment. And so to have a component of our programs or our systems that are then looking at data as a product to say, once we deliver that sensor or that um analysis capability, there's still a consumer somewhere out there, whether that's in the service, um you know, on the ship in the service or especially now as we think about joint old doma command and control in the joint force and without and partners who may need that data and wanna be able to use that data, we have to shift culturally to be able to have. I think somebody mentioned it earlier, those data product managers um to be able to uh have that producer consumer relationship. So, speaking of data as a product, what is your office? What's your office's plan to move the department into that direction? Yeah, so we have um under CD A O we um we are the chief data officer for the department. So that's one of the, the hats that we wear. Um and with the other chief data officers across uh the services and the commands um as well as um at the OS D level. Um There are data officers in each one of the principal staff assistants. And so this is like um OS D personnel and readiness acquisition of sustainment, research and engineering. And, and we have ceded data expertise at each uh one of the P S A areas as well as at the Cocos. The idea behind this was to give the chief data officers at these organizations, the talent that they need to really understand as both data producers and consumers, how to manage the data that that supports their decision making. Um One of the principles of kind of data mash, which we're we're looking uh to implement is this idea of domain centered uh uh data management and architecture. And so by functionally aligning uh the data experts in these different offices, we're now able to reach into dod across all those different communities. And start to say, hey, you know what day is important for you to deal with? The logistics community has done this incredibly, Chris o'brien and the J four and her team as well as the rest of the J four J four logistics community across D D. As you could imagine given recent world events between COVID and the Afghanistan withdrawal and support to Ukraine has really embrace this idea. They have gone through all of the data that they use to make decisions. Um They prioritize that they've gone after every single database that houses that data, they prioritize that they put in place modernization plans for the I T that supports it. And you really start to see that whole community kind of evolving around a data centric approach to decision making. The CD A O. Craig Martel has described a hierarchy of needs, right? So uh getting the data right first is at the bottom, then the analytics and then A I at the top, can you explain the thinking behind that. Yeah. Um Lots of people love to talk about A I. Um we love to talk about A I. Um And uh it, it is prerequisite upon certain things. Um You, you absolutely need the cloud computing capability. You need the talent that's gonna have digital literacy. Um You need the acquisition strategy. So it's gonna allow you to get out of that system specific mindset and into more of a software or algorithm type mindset. And so those are the enablers down the bottom. Some of those in CD A O uh or some of those CD O does not own. Um So good partnership with the cio um honorable John Sherman uh on software modernization. That implementation strategy just came out. We're really excited about that uh on the cloud computing aspect, which we're finding cloud capability, really excited about that. And a bunch of a bunch of those different enablers. The next step up is, is quality data and we're spending a lot of our time focused on how do we improve the quality of beauty data because without that you can, you can dashboard and A I all you want. Um But the recommendations and the insights are going to be limited if, if the data is not high quality. So we're doing a lot of work there uh on data cataloging on identifying, you know, where data exists across the enterprise um creating uh in, in certain pockets, certain communities onto for how to do that in a common way and look, look at things uh describe things in a common way um as well as put in place, some of the architecture is necessary for data access. Um Next up is the analytics. Uh dashboard really helps us understand kind of a view of ourselves. Uh I think uh I worked for Chairman Miley for a while and, and he would like to say, you know, I just want to see what we have, like, show me all the stuff we have. And so um Greg Little will call himself the chief counter, you know, can, can we actually count all the things we have and in dod and, and ana analytical dashboards, give us that sense of um what that looks like really good work happening in the strategic readiness realm right now with uh getting into predictive readiness. And how do we understand um how to not just reflect our current status but project forward and, and build models out to do that? I know the services are doing really great work in that area and then A I at the top. Um And so now it's now how do we take these um this understanding of, of who we are and what we have and what we wanna do and start to aid uh human decision making uh with different algorithms that will give us insights that um we couldn't do as fast uh as, as the machine could, let's talk a little bit more about that analytics layer because, you know, the Dod has massive amounts of data, some of it useful for decision making, some of it not so useful. So what's your, um what's the role of your office in ensuring the usefulness of the data for the end user? Yeah, absolutely. Um So it's a team sport. Um I think, and I think over time you were asking about the history before, you know, in the last five years or so, Dod has uh rightly, so I think I've, I've read almost every uh consultant article out there on Digital Transformation. Um But, but rightly, so taking kind of a use case or pilot based approach to understanding how to, you know, digitize ourselves. And we've learned a lot about different communities, what's in the realm of the possible. Um And, and what is right to do and or what is best to do in the case of our culture and our organization. Um But now we've learned those lessons and it's time to scale. And so, uh Ivana, for example, has been an incredible platform for people to come in, start to design uh things that meet their needs um on a kind of use case by use case basis. And as more people have come to that platform, uh really with the big support of a core CD O team, kind of helping with that data access, with data clean. Um definitely in support of the secretary and the Deputy Secretary's, you know, high priority use cases. We have a role in taking those very um you know, leadership driven use cases and, and helping with the data access and visualization analysis there. But downstream, it very much is, you know, something that can go all the way down to the citizen data scientist, some somewhere in dod um to be able to pull up whether they're in or whether they're um at a Fleet Readiness center or wherever they might be to, to use that themselves. And, and then that is a very local uh or community driven type of activity. Um But as we think about how to scale, we want to put in place those enterprise pieces like the data catalog, the model catalog. Um and the ability to really have a self-service data platform through van um that can, can meet all those needs and then democratize the ability for others to go in and um develop what they want. How are you thinking about issues around data sharing and data ownership? Yeah. Um We don't think there are any data owners in dod. Uh The deputy secretary uh released her data decrees that basically said the secretary and, and her are the leaders of uh or the owners of data across dod. Um And everyone else can be a data steward uh or a data product manager. Um And for the most part, um in our Chief data Officer hat. We have never had to go to the Secretary of the Deputy um to force state access. Um It's, it's a nice card to have and, and, and to wave up and down every once in a while. Um but most of the time, um what we real, what we find is that um there are layers in between the person asking the question and that data steward and in the middle, there's a lot of concern around. Why do you want my data? What are you gonna use it for? Aren't you gonna tell me what to do if you get access to my data, people at dod are very busy. Um And so we don't generally dive, dive into the um the user's business unless it's related to, to something that's usually a higher integrated question. Uh So may some may push back on that. Um uh But um the, the ability to have data stewards who understand their data the best. Somebody said, owner said it earlier. If you're the creator of that data, you're probably the expert in it. Um And that's what we want to really cultivate is that um we have these data product managers, these data stewards across the enterprise and uh they're able um to understand customer needs wherever they exist out uh inside of Dod. And we'll be able to take a couple of questions from the audience so you can start thinking about that. But I want to talk a little bit more about sharing because when you share data, a lot of issues come up, there's privacy issues, there's legal issues, there's technical issues. So how are you thinking about all of that? Yeah. Um There's, there's a couple of ways um without a doubt, there are protections in place for things like personal information, health information, uh privacy, civil liberty security. Um and, and we adhere to all the federal regulations inside of Dod and that, so we're not, we're not outside the bubble uh there in any way. Um We want to make sure that we're gonna deal with that data effectively, especially um you know, in light of recent events, um the Vanna platform in particular has identity based access and so you log on for to an account, you can see the data that you're authorized to see. You can't see the data you're not authorized to see. Um And that has really given us an opportunity to create communities or, or make sure that we are controlling the access to information for people that may not have a need to know or need to see that, but also cultivate the sharing across the different groups. We tend to defer to the data um provider on who can, who should see that data and then we'll broker conversations. Um if there's some concern that where one party wants to, to see it, but the provider doesn't and then we can, we can have those conversations and, and determine if the need is necessary. And, and finally, Margie, before I open it up to questions, obviously, there's a lot of industry partners in, in the room and, and online. What are you looking for from the defense industrial base? Yeah. Um A to a ton. Um And I'll, I'll count it as a two way ask. Right? Because I, I think I heard before on the industry panel, some comments on this. So we know inside of CD A O that access to government data is really hard for industry. Um And we are working really hard to try to expose our government data to industries, particularly for software developers um or tool, you know, other data tool providers um to be able to come in and have access to at least a data set where they can show the showcase their capability, um train their capability uh demo their capability um in a way that um you know, make sure that we have, you know, control over that data and, and keep that secure, but also, um you know, creates opportunity for industry to do more with software. Um Some of that's tied into our A T O process. That's why we're really excited about that software modernization implementation strategy from the cio that I talked about to get to that continuous delivery of software and around the security hurdles. Um We also have a couple of acquisition vehicles. Uh One is called try A I, it's specific to A I but it's a no cost vehicle, but it contains all of the protections of industry. IP um As well as government data rights. We provide the data source, industry provides the tool and we can have a conversation on how does it work? What does it do? How would we use it? Um Is this what we're looking for? No, we're looking for something else. Um And, and then the team can uh decide if they want to go to an acquisition vehicle or, but it, but it gives a chance to provide direct feedback. And so we're looking at opportunities to increase the conversation to provide access to more government data, open up our platform so that um we could have more commercial vendors develop on government data and then back to that data as a product piece. Um ask industry that if you're going to provide an analytic capability that creates new insights out of data, there's a way to provide that those new products back into the enterprise to be used by other applications and, and data uh data analysts. Um And so we're thinking through whether that's contract language, whether that's tooling, uh whether that's services inside the data mash. Um But really one um a couple of things from industry feedback on how we can be a better partner as a government. Um the ability to think about uh your capability as part of a broader data mash ecosystem um where producers and consumers are going to be very diverse, just like D U D is really diverse uh and decentralized. Um And then uh specifically in the A I realm, probably more, but um we're going to require some different things on the government and especially the operator side um on from A I that the commercial sector maybe may not require uh just to provide more assurance uh for, for our emissions. All right. Well, we'll start taking questions, um start over here. Hello, I wanted to ask you, what are your thoughts about interoperability and compatibility both between services and with legacy systems within a service? Yeah, absolutely. I spent a large part of my career in Navy on integration operability. I and I type things Navy and Great Fire control counter air. Um really, uh you know, decades long efforts to get systems to talk together. I think there's a difference in the, there are unique problems at the tactical edge, not gonna hand wave those that absolutely come in. Um Because they are so connected to connectivity um where we are in CD A O is trying to think um specifically on the, on the warfare side um at the combatant command level, the joint level, how do you integrate across combatant commands? Um We run an experiment called the Global Information Dominance uh experiment series, uh which we took over from north com that looks at how you integrate data across Cocos globally. Uh We ran our first experiment back in January and got a lot of lessons around data interoperability both um within a community. So I talked about the J four before they were one of the participants. Um How did their work work flows work within a Coco? And then how do their workflows work across Cocos? And then how does the four workflow work with the intelligence and the operational workflow? And so we're testing out all those connections in the context of these workflows because that's where the concept of how we're going to operate and the data comes together um lots we can do in a cloud based environment um that, that we can immediately put in place and we don't have to wait for those, you know, unique cases at the edge. Um But we're working closely with the services on how they connect into that uh when connectivity allows. Um And, and we're looking at this kind of data mesh and how you create the services for transformations of, of different data types. Um But again, back to using more code, less uh translation code, less standardization, less network based interoperability connections. Um I think we have a lot of work to do there with experimentation uh to figure out the right balance of that other questions. Yes. So I guess the question that everybody was waiting to ask is all about the revolution and gender of A I, right? Um So how does the CD A O um take bring in the tools from general of A I chat GP T and all those in? Um And then what are the risk you see or how, what do you ask from us as industry and consultants to help you incorporate these new janitor of A I tools into the dod uh for your work? Yeah. Um So I'm gonna caveat um that I am not an A I expert. That is, that's my boss, that's our team. Um I've got the, um the, the institutional expertise for better or for worse um on, on how to, you know, take these kind of new emerging fields and, and, and get them into dod uh culture. But we are having an intense conversation inside the CEO about L L MS. Um And I'll just give you a couple of threads. Uh And then I'm gonna have to toss the, the more detailed questions over to um Joe Larson or, or Craig Martel. Um When it comes to dod applications, we make decisions that have significant consequence, whether it's in the budget space with taxpayer dollars or whether it's in the warfare space with uh potentially, you know, lives of, of Americans and others. And so, um for administrative purposes, um you know, you might think, oh, this is a great tool but, but when, when the risks of uh fabricated information uh can, can come out of a tool like this, we have to be really certain that the conclusions that are gonna come from the tool are grounded in uh traceable, uh you know, reputable sources. And so we're thinking through, you know, how do you uh understand how the content is produced from an L L M and make sure that um there's a, an ability to make sure that um there are the right hooks in so that we can have confidence in what it's recommending. Um You know, people are not uh the editing process has, I don't think anyone's favorite part of uh grad school uh or undergrad. Um And so when you get an answer, it's very easy to, you know, just kind of say, oh yeah, that looks good and ship it. Um But we're gonna have to also think about training our workforce force on when you get this answer, there's still human work that needs to be done to make sure that um it matches your cognitive understanding of the context and the situation. Um And, and what to use it for. And so, um yeah, we're, we're thinking very heavily about, you know, how do we bring this type of capability into dod? We see massive potential uh use cases inside of the government, both on the administrative side and on the operations side. But we want to do that responsibly and make sure that we've got the confidence uh in those tools when we, when we use them. Well, we're out of time, but I'm gonna put in one more squeeze in one more question. And that's about what's your biggest priority for this coming year? One that your top priority? Just one man. Um So of all the things that we're doing, I think um it's a huge opportunity to have a chief uh digital and A I office uh which looks exclusively at data at Analytics A I as a direct report to the Deputy and Secretary of Defense. Um And so the priority is really to establish our agenda and our team to be able to continue this work in an enduring way, regardless of personality, both on our team or in our leadership. Um And make sure that we've got d dod down the right path and that we're not just doing kind of things in the traditional um way we've done them before, but we understand where the barriers exist and which ones we have to remove to be able to really go after this capability effectively. And I think instituting that ground work um this year is, is really critical to our success uh going forward. Thank you so much. Nice to have you. Thank you so much. Thanks for having me. Thank you.ÌýÌý