Pre Submission Ask Me Anything for Multi-Decision Advantage Sprint for Human Machine Teaming (M-DASH) Event.docx

Pre Submission Ask Me Anything for Multi-Decision Advantage Sprint for Human Machine Teaming (M-DASH) Event-20261015_115215-Meeting Recording October 15, 2026, 3:52PM 1h 41m 39s

TOMPKINS, KALEB G 2d Lt USAF AFMC 711 HPW/RHWID started transcription

ProWave AI - Reid Webber 2:17 Yes.

Jon’s iPad 2:51 Doctor frost. It's be doing a sound check.

My Internet literally just came back.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 2:56

That's close, but I got you loud and clear.

Jon’s iPad 3:00 OK, I'm gonna try to see if I can push slides from my computer.

Do you mind if I just try that and we can put yours right back up?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 3:07

Yep.

I can see your screen.

Looks good.

Jon’s iPad 4:21 OK, I'm gonna just try one thing here.

OK, OK.

Thanks.

Serena Hacker 4:57 This is the amdas opportunity.

I'm just gonna record it so we can talk.

David Callner Uandor 7:27 I needed to change a pin.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 8:16

Hi everyone.

We're right at 12.

I'm gonna give it another two minutes though.

Since there's still a lot of people calling in.

OK.

Welcome everyone.

So I'm doctor Elizabeth Frost with the Air Force Research laboratory.

I go by Z, so I'm going to step through a few slides today covering the call for the upcoming multidecision advantage Sprint for Human Machine teaming or M dash.

This is very much AQ and a session, so although I will be talking through a few slides to hit some of the major key points.

Absolutely. Feel free to raise your hand.

Or interrupt at any point with any questions you may have. Once I get through an overview of the event, I will be handing it off to John Saul, who will be giving a deep dive academic session over the problem space that we're looking at for this. So like.

I mentioned I'm with the Air Force Research Lab, but this is ran in collaboration with the ABMS CFT and the 805 CTS.

So I want to do a quick recall and make sure who's on the line.

So do we have a veril contracting?

On the line.

EMMERICH, KATELYN A CIV USAF AFMC 711 HPW/RHWTE 11:30

Three is running about 10 minutes late, but you can press.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 11:35

An RRFR Amanda.

SZCZUBLEWSKI, AMANDA L CIV USAF AFMC AFRL/RRDE 11:39

Yes, I'm here. Hello.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 11:40

OK, great.

Anyone from the ABMS CFT?

STALLINGS LILLY, TEINA D Col USSF HQ DAF HQSF/ABMS CFT 11:48 Good morning.

There are several of us online.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 11:50

OK.

Great. Thank you.

And the 805 shock.

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 11:55

Doug Hayes and LTB.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 11:57

Great. Thank you so much.

OK.

So this is a lot of the leadership team for these events.

So there's a few important dates.

Currently it's 15 October, which is this pre submission A May. So you are here coming up. Then on 21 October at 1700 Eastern submissions are due.

You then selection notifications will be going out approximately.

The 4th of November.

It's usually a couple days around there and then the event itself will be held 25 January through the 5th of February out in Las Vegas, so PM contracting.

They're joining here shortly, but Kate Emerick is on the line who can cover some of it.

Kate, I didn't know if you want to step through a few up front points regarding the submissions and the dates.

EMMERICH, KATELYN A CIV USAF AFMC 711 HPW/RHWTE 12:56

I I think you covered them.

There's just some contractual, you know, statements here, right?

Once you submit, please make sure that you receive a confirmation e-mail and if you don't reach out to those contacting POC's that are listed on the call.

Ben Sparklin 13:08 You'll see is a.

EMMERICH, KATELYN A CIV USAF AFMC 711 HPW/RHWTE 13:13

The AMA slides from today and we will post Q&A Excel spreadsheet from any of the questions from today will be posted on sam.gov.

If you've already submitted a proposal.

You are able to amend and resubmit that after this AMA up until the proposal deadline and any proposals received after the deadline won't be considered or evaluated. If you have any additional contracting questions. Again, reach out to those POC's that are listed on the call. Thanks.

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 13:46

Hey, this is Doug Hayes.

Can we get a copy of the slide deck for eight? Oh, fifth continuity.

EMMERICH, KATELYN A CIV USAF AFMC 711 HPW/RHWTE 13:51

Yep, we'll make sure you get those.

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 13:53

Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 13:59

OK, so to give an overview, a high level of the M dash. I'm assuming everyone has read the CU at this point, but high level.

We are looking at the battle KOA ensemble, so if those have heard of dash events before, they've looked at single decisions. This upcoming dash will actually be looking at an entire battle KOA ensemble.

It's broken down into three specific decisions.

That John saw will walk through later on, the first being perceiving an actionable entity.

2nd is matching effectors and then finally generating battle COA.

So it will be expected that each proposal or group provides services that do all three decisions and I'll get into the expected deliverables from these services.

On the next slide, but the high level point is that although it's broken down to three decisions.

We are expecting the complete battle KOA ensemble to be demonstrated by each accepted vendor. So the reason we're looking at these decision making services is that the C2 systems today are focused on information advantage, getting the data where it needs to go, visualizing that data, but the.

Decision making is still really falling on the human, and so when we talk through the solutions that we're looking for as part of this event and the proposals.

We're really focused on the decision making.

Logic and reasoning behind the services itself.

I already mentioned this is a collaboration between AFRL, the ABM SCFT and 5th. The events we host out in Las Vegas are over two weeks. The first week is really to focus on giving a deep dive of what these decisions are working with the warfighter to understand the.

Problem space.

Understand the scenarios and understanding.

The difference between information and decision making and the reasoning behind that a big part of this event will be making sure we are getting feedback on the actual logic and decisions being made.

I'm not the display or visualization of it, so the vendors are not expected to provide a user interface, just the services that will be plugging into a common user interface.

Doreen Drescher 16:16 Saving us.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 16:25

So week 1 Deep Dive initial software coding.

Understand the problem space the second week, then is where we're due initial progress checks and then towards the end of the second week, each vendor will be expected to run in at least one one hour scenario with their tool being used by the warfighters themselves, so to EV.

The effectiveness of it.

Which leads into the deliverables so the expected deliverables from this event are that demonstrate functional relevance like I mentioned, being able to provide a software service that contributes to enabling human operators to make these decisions better and faster.

As far that the outputs of the software services created are expected to be provided to the government software System API, using the government provided data schema, we will not begin to that now.

The details.

Of all that information will be provided post selection.

We also will be looking for traceability logs, so since three services will be running, one for each of the three decisions as part of the ensemble, we will be looking for how those services interact together underneath the hood and just calls between the services.

Then at the end of the second week, there will be a government only out briefing.

So you will be asked 2% in power.

Format your event overview, the tech approach, relevant finding metrics and feedback.

This is a one-on-one chance with the government to try and protect OIP, but for us to get deeper understanding of the work done, we also will be providing a technology survey.

We will give more details about it again post selection, but just understanding AI models utilized a lot of the hardware and runtime requirements for us to understand.

These systems forever to be deployed on what the requirements would be for that.

So I'll pause here for a second to see if there's any questions regarding the overall event or the deliverables, and I'll answer them to the best of my ability. I can at this point in the process.

Yes, Kurt, from kakki.

Kurt Hungerford CACI 18:49 Hi. Hi doctor Z.

Yeah, this is Kurt Hartford from CACI.

Did I catch you right saying that you were not expecting AUI from us this time around?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 18:59

Correct. So we since there's three decisions services that need to be developed and tested, there is a government owned UI that will be used. So as the the requirement of that output being provided in the API that will actually feed the UI.

Kurt Hungerford CACI 19:18

OK.

So pretty much if we have that met, it'll be up to talk to you guys to kind of however, you're gonna present it and everything.

We don't have to worry about that this time around.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 19:29

Yes, absolutely.

Just on making sure the data is in the correct format and then it will be presented.

So there's one UI used for all the runs to keep that common.

Kurt Hungerford CACI 19:38 OK. And and are do are are you will our logs?

Those trashability logs are also. Is that just something that will kind of also?

Doreen Drescher 19:46 It's all but.

It.

Kurt Hungerford CACI 19:48 Submit to to an end point of yours as well. Or or are we going to be you know, is there like an expected?

Log sheet or something that will kinda output and and submit to you guys at the end of it.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 20:03

Yeah, we're work through how that exactly works.

Probably post selection, where do some means beforehand.

I think that's working out, but the government team the A for L team especially is on hand to help make sure that we are able to get all the data in the format and so I don't have the details for all that right now, but I will have.

Them and we will absolutely be on hand to help so that this isn't something holding you guys back.

Kurt Hungerford CACI 20:29 OK, all right. Makes sense. Thanks.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 20:36

Terry.

Terry Guild 20:41 Hi Doctor C.

It's Terry Gout from intrinsic. Thank you.

Just a quick question. You know we we realized that these there's been multiple events over many months.

We're just wondering if the M Dash event is kind of based on prior work, and if not, you know how can potential new vendors participate and if there's any lessons learned that are shared with industry where we could be a little bit more competitive to pitch from you.

Know from a capability standpoint?

Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 21:10

Yeah. So I'm sure Kate will step in.

You know, I have to be a little careful about what I say with everything but.

We some of these decisions, we're looking at this one are previous ones we're looking at.

So we ourselves running these events have learned.

More and kind of refined and grown on the execution of these events.

And so there's not, I guess.

There's probably a lot of lessons learned I could share, but really the way we've set this up and the way we're running it is to make sure that all industry partners have an equal chance.

And to give opportunities based on the things we've learned to share that knowledge.

So I guess to kind of half answer that question is, yeah, there's a lot not a lot I can share right now.

More so that we have, we're doing this to make sure.

There's more opportunities as these events have grown as the news has spread about them and we are the Sioux itself and other information we're providing this AMA are ways we are sharing that knowledge learned Kate.

I don't know if there's anything else you want to step in say about it.

EMMERICH, KATELYN A CIV USAF AFMC 711 HPW/RHWTE 22:29

I would just say all proposals have right.

We are.

We are looking at these proposals for this specifics sooo that is put out and there's no prior.

Dash events that have a way on the technical evaluations that are done on these current SUS.

Terry Guild 22:52 OK. No, I appreciate that's that's helpful context for us. You know, we've been selected but not funded and we didn't know if we were just behind the power curve.

EMMERICH, KATELYN A CIV USAF AFMC 711 HPW/RHWTE 22:52

Over.

Terry Guild 23:01 But I appreciate the context. Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 23:11

Yes, David.

David Harris - IDS 23:14 Yeah. Thank you.

David Harris with Integrity defense solutions.

Just maybe following on the last one, is performance going to be measured against a a standard like benchmark or or prior?

Tool? Or is it measured against the performance of the other teams selected?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 23:33

No. So um.

Yes, this is not a competition. These events we state very clearly up front.

We do not see these as competitive events at all.

What we are doing in week one of the dash events is collecting a baseline measurement of how the warfighters do it with no machine assistance at all. So that first week.

That baseline battle management scenario, saying how many decisions are they able to make.

The speed a few different quality measures of those decisions of the humans alone.

And then we are looking at the same metrics as they use each solution. But one thing we're interested in too is just.

There's not always a quote, quote, best, right?

There's different approaches, and So what we're trying to understand too is what's the trade-offs of different approaches used in different models. And so really these events aren't to grade people's homework, so to say as more so for us to understand what industry has to offer.

And the variety of the solutions out there when going into it, if that makes sense.

David Harris - IDS 24:47 Yes, thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 24:52

OK, Jeremy.

Jeremy Gordon -Avix 24:55 Hi jeremi.

Gordon. Hey, mix, I guess along the lines of the previous two new vendor for this, what is the expectation on software completeness upon arrival versus how much is like iterative development in that second week?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 25:12

Yeah. So we are not expecting anything on pawn arrival.

I might have.

Kate has to step in, but I believe the the the performance of period is just for the two weeks, correct Kate?

EMMERICH, KATELYN A CIV USAF AFMC 711 HPW/RHWTE 25:26

That's correct. Yep.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 25:27

So the only thing we're expecting is that for over this two weeks for this event that regardless of what state it comes into by the second week, there's just a usable solution.

We do not have any sort of expected.

Level of maturity.

It just has to be something we are able to demonstrate the functional relevance of it within that second week.

Jeremy Gordon -Avix 25:53 Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 25:57

Aaron.

Aaron Benlolo 26:03 Thanks doctor Prost.

I just had a quick question.

Aaron Bolo from Mission Point.

In terms technically of what might be available during the event.

Are our vendors expected to bring their own models or are you going to have models available that are local? Or will it be Internet access to frontier models?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 26:22

Yeah. So I will.

I was gonna dive in to this next little bit with the facilities, but I can say right now we do not provide any models or tokens.

So any that are leveraged will have to be at the discretion of each vendor themselves.

Aaron Benlolo 26:47

OK.

Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 26:48

Yep, Adam.

Adam Dolin - Z1 26:52 Thank you.

It's Adam Dolan with Z1. I guess now I've got two questions.

One, I guess in a perfect world.

Would the solution cover all three of the PAPAEMEF and GBC?

Or would you prefer that?

It is sort of like 3 segmented solutions. I guess ultimately the results would go to the same place, but does it does that kind of matter at all?

And then two, if we are choosing a model to plug into a system.

Is it preferred that we use an approved model off of the latest approvals?

Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 27:37

Yeah. So we have no restrictions upon the models used at this event.

And so that's up to your own discretion if you want to try out different models and kind of look at the cost benefits of it, that's also very welcome.

You know, we're always interested in information like that, but absolutely not required.

So the model again is completely up to your own discretion in terms of.

The structuring of the software itself, we just wanna make sure that three separate outputs.

Are provided, so we will need an output for PAE, one for MEF and one for GBC. How that's exactly structured, but behind the scenes we are not dictating for this event at all. We just wanna make sure that we are able to get.

Three separate outputs for each decision, and not just a final one at the end of the ensemble.

Adam Dolin - Z1 28:32 OK. And then?

So if we were connecting to an API, is it preferred that those 3 outputs go straight to the API feed or?

More like save to a Postgres or a Ledger and then you know that's immutable.

Send it on through.

Is there a preference there?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 28:55

Yeah, we will want it since the outputs is what will be driving the UI during the scenario execution. We will want to be sent directly to the API, Yep.

Adam Dolin - Z1 29:05

OK.

Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 29:11

Levi.

Levi - CACI 29:15 Yes. So I was wondering cause yeah with the three decision functions being happening at this one.

Is it more targeted at like the chaining of the three decision functions and how they guess they work together and complement each other? Or are we gonna be more focused on evaluating the quality of each individual decision function?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 29:35

So we're mostly looking at each individual decision function more so than the interactions necessarily form during this, however.

I don't want them.

We're not gonna look at that, right?

But we will be looking specifically at the.

Outcome of each individual decision.

Levi - CACI 29:57 Sweet. Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 29:57

Yep.

Alex.

Alex Olihovik (CACI) 30:03 Hey, this is Alex Olahupe from CACI before the M Dash begins. Will we have access to the government software System API so that we can prepare how our solutions are given to you?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 30:12

Yes, post vendor selection. We since it's not technically during the pop, we have optional office hours that will we will be going over all of that.

It will also be available on day one and at the event there is an AFRL group that is on hand to help out with it as well.

Alex Olihovik (CACI) 30:35 Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 30:41

Someone from pytho.

Akshay Mittal (Pytho) 30:44 Hi, Akshay, from pytho.

This time, are we gonna have vendors doing all three services or is it?

Is it possible that we only get a subset of the services?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 31:01

No, each.

Vendor or group each proposal they that group needs to be able to provide all three services.

We will be running each of vendors solution on their own.

We won't be mixing and matching decision services across vendors.

Akshay Mittal (Pytho) 31:23 Got it.

Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 31:27

Aaron.

Aaron Benlolo 31:31 Yeah, Aaron, from Mission Point again.

Is there a minimum or maximum group size you expect for the event for a per team?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 31:40

Uh, no. There is a space limitation, so we can get into the details of that.

We have seen, you know, 3:00 to 6:00 to 8:00.

So it it really varies pretty greatly we would ask.

Please don't bring a team of like 15 people because the space limitations, but enough to be able to be there to deliver the required deliverables at the event.

Aaron Benlolo 32:13 Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 32:13

Yep.

Terry.

Terry Guild 32:17 Yeah. Thanks again.

Just a quick comment and really question the the statement of objectives document that has been published.

The links are kind of broken.

They're they're confused.

So if you go to the PAE link, it actually takes you to the MF related dash.

So I was hoping we could get an updated Sue that has the correct links in there because what we're missing now is the PAE dash link. Thanks.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 32:43

Yes.

Kate will make an Yep.

EMMERICH, KATELYN A CIV USAF AFMC 711 HPW/RHWTE 32:45

I'll take a yeah, I'll take an action to check on that. Thank you.

Terry Guild 32:49 Thanks.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 32:55

Jeremy.

Jeremy Gordon -Avix 32:57 Yep, one more question.

I know you wouldn't provide the API details until after selection.

Is there any expectation going into this for?

The ability to ingest specific data types like from a JRE or Mace or something like that.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 33:17

Yeah. So that actually kind of leads me into my next slide.

So I will probably lean on the 80 fifth here in a second to give anymore overviews or answer some of these questions, but I'll give a high level overview of it. We do have a various data sources.

I we will get into that again post selection. The idea though is there's different ways to access it.

We have over, you know.

So.

The long time running these events tried to make everything as ingestible as possible.

However, there will be expectations that potentially data translation layers are needed, right?

But post election, we will provide details about.

All available data sources.

Upfront there will be text documents.

We do have formatted database that the contents of those are accessible from. There's an IRC server.

And then there is other track area entity data that's available through rest assess in other ways, but the details of that will be provided post selection.

So at the Facilities, Theater, 5th CTS or Shock group host us, they are an absolutely fantastic group.

Who has done so much work to make everyone's lives as easy as possible, but just some awareness?

Before the event so that there's no confusion or worry about whether or not your solution will actually be usable out there. It is an unclassified environment.

So there's absolutely no classified materials or discussions at these events?

Everything is at the distro A level. We do have international allied partners that attend these events and we wanna continue to have them without the worry about the extra classification levels.

Each selected team will have their individual development space on these aren't large spaces, but they are kind of cornered off areas for you to work with.

A shared area where the crew operators will be running and we can test out software out there. There is some hardware on site that is used at the cruise stations the Warfighter stations.

That hardware, though, is only to be used for the scenario an operator use. It is a bring your own hardware development environment for developing and then hosting and so through a web page or some other.

Application is how we prefer to be able to reach to your service.

There are ways is very, very complicated and a little bit of a hassle to be able to deploy your software services on the hardware on site itself.

So bring your own hardware.

The 805 fits online to answer more questions about that.

The networking is commercial Wi-Fi. There is local network for data connections at the event, but all of the Wi-Fi there is a commercial.

Open Wi-Fi.

I know there's sometimes questions about reaching back to companies, servers and VPNs, and that those are things that have been worked through, and I know.

There are solutions for, but I will stop here and hand it over to anyone from the 805 that wants to clarify further answer. Any questions regarding the setup of the facilities at the event.

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 37:08

So most people, one of the very first questions I get asked.

First of all, I'm Doug Hayes.

I'm one of the leads for eight. Oh, fifth, along with Lieutenant Bartow and Staff Sergeant Clark is power.

We will have power installed, you know, with little.

What are those little?

You these.

Power strip.

Sorry loss for words.

But we ask that you do not go directly into the wall and we will be available to help you get your power situated in your room.

That's 'cause wheel trip circuit Breakers, and then we'll take the whole exercise down South. Power Distribution is one of the top things on our list.

But as far as networking or any other questions, please ask.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 38:10

Keith.

Citrenbaum, Keith I CTR (USA) 38:12 Yeah, Keith, Citron, Baum with Sosa Corp.

If can we host on like something like AWS where we have our development environment currently hosted and that type of stuff. So you can reach out to a web service over the Internet and then receive information back. Or are you expecting everything to live within the local network?

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 38:40

Hmm.

I do not see a problem with reaching back, but if you're doing like a masses amounts of data transfer that could stream the network.

And we do have about 100 people every day here.

Citrenbaum, Keith I CTR (USA) 39:00 Oh yeah, it's not so much on the amount of traffic. I'm just.

The local client machines that are gonna be running stuff in the APIs you're exposing.

Are would those for example be accessible from a external hosted environment, either an AWS or a corporate environment where our services would actually be running? Or do you expect all of the local all the applications to run locally?

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 39:29

So let me make sure I got this question in my brain the right way so.

You're talking. You want to connect like a remote system into our exercise network.

And be remote.

Is that what you're asking?

Citrenbaum, Keith I CTR (USA) 39:50 Well, no we won't.

We would be there physically on site, but instead of having to bring a hardware stack with us to host our applications, our applications are hosted in a something like AWS for the local kind of execution network.

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 40:11

Can you repeat that question?

Citrenbaum, Keith I CTR (USA) 40:12

OK, OK.

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 40:13

One of my guys online.

Citrenbaum, Keith I CTR (USA) 40:13

OK.

So the the network topology there, right?

We have, you said there's a commercial Internet for development, right?

And that you expect?

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 40:27

Just a minute.

Citrenbaum, Keith I CTR (USA) 40:28 The vendors to be self hosting in one way or another, the applications that they're building or bringing right.

And that network I'm assuming would necessarily be connected up to the API endpoints that your systems are running on.

And then your systems will be able to call the API endpoints that our system is exposing.

The question is, could the systems that we're hosting be hosted in an AWS environment where your systems would be connecting or you're hitting those exposed endpoints versus a local machine that's on site?

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 41:09

Hey this is.

BOWMAN, SARAH E CIV USAF AFMC 711 HPW/RHWTE 41:10

It.

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 41:12

Oh, this is Mike Shin.

Yeah you can.

At least physically, that is definitely. I think that can be done.

Citrenbaum, Keith I CTR (USA) 41:19 Yeah. Obviously we'd have to work out all the the routing and VPN ING and stuff like that, but OK, good.

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 41:25

Yeah, it's it's physically a commercial network.

With very little firewalls between US and the open Internet.

So yeah, anything that should work anywhere else in the world should be able to work here too.

Citrenbaum, Keith I CTR (USA) 41:41 Perfect. Thanks.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 41:59

David.

David Harris - IDS 42:02 Yes, thank you, David Harrison, IDs again.

I'm just following up on Keith's question basically, but just to clarify and be crystal clear for the models I heard what was said earlier about, you know we we can use whatever models we want and and multiple models.

Is there any restriction on the scenario data the the the API data that we can send to the model?

Is there gonna be any restrictions there?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 42:37

No any of the data available and streamed is all destroy.

David Harris - IDS 42:46 Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 42:57

OK.

Any other questions regarding the facilities or the event itself?

And if you guys think of anything else, feel free to reach out to contracting, whose contacts are on cm.gov, and we will be posting AQ and a sheet that we will update with questions too as they come in.

Chet.

Chet 43:21 Yes, ma'am.

Thank you so much.

Just a question around the the data types in a more technical sense.

Can you provide some idea in regards to whether it's structured data, unstructured data, whether it's streamed or batched or just flat files?

And what volume, volume and velocity?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 43:41

All the above.

Yeah. So there's track data, Erie data, there's some data.

So, like commander's guidance plans, some of that is all provided normally in text documents. Like I said, we also have it available in a database that is kind of the starting situation.

That doesn't really change.

Then there is dynamic things that will happen throughout the scenario. So.

A track position updates and other things such as that.

There's also a lot of unstructured data, both chat and voice comms.

We do have a speech to text capability that is fairly reliable and so really it's all chat, ultimately unstructured. But the way voice comps and chatcoms are can be different.

There is some afral support tools that have been developed to try and.

Greatly reduce the burden of parsing this data. What we don't want to happen is.

You guys were out there and spend half the event just trying to consume the data in a way to be able to reason over it.

So a lot of different data. I'm in different sources, but again, Afro team has done a lot of work to try and make that data as consumable as possible and we'll be out there to help any of the vendors consume that data as well.

So be prepared to make sure your solutions are able to kind of consume and reason over it, but trying to get that data.

Data in a way you can consume it.

We've tried to lessen that burden.

Chet 45:20 Perfect. Thank you so much.

And then just a quick second question as well.

Are there any structured frameworks or doctrine that you are abiding by or required to follow in the decision making process that we have to model in in our responses, in our logic flows?

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 45:41

I will let Johnsall kind of go over that when he does the academics more so for these decisions.

If beep.

Chet 45:54 Perfect. Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 45:58

Doug.

HAYES, DOUGLAS E CIV USAF ACC 805 CTS/DOZ 46:01

Yes, ma'am. I just wanted to cover one thing.

We have a Wi-Fi network that you'll be allowed to connect your cell phones to.

Where we run into a discrepancy is connecting your cell phone to the exercise traffic only network and then linking your cell phone and your work computer together.

So we ask that you not do that.

Keep your Wi-Fi cell phone on the Wi-Fi only network and your exercise traffic.

On the Wi-Fi network designed for exercise traffic, which will put all that information out again once you're on station. Thank you.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 46:58

OK, if there's no other questions at this time, I will stop sharing my screen and hand it over to you, John Saul, but again.

Absolutely. Feel free to reach out at any point between now and the submission deadline to either contracting or myself, although it's probably going to be a quicker response through contracting than that, but I'll hand it over to you, beep.

Jon’s iPad 47:25 Great quick sound check for me please.

FROST, ELIZABETH M CIV USAF AFMC 711 HPW/RHWTE 47:28

Got you loud and clear.

Jon’s iPad 47:30 OK, terrific. And I'm, I'm operating on two devices here.

So if I start to lose connectivity, I may have to switch between them.

My name is John Zahl.

I'm supporting Dr. Frost and Miss Sablesky on the C2ES project, helping to.

Basically, build the government's reference model about decision making.

Something we call the transformational model for decision advantage and then working with them for these experiments.

And first of all, thank you very much for being here.

We're really excited to have the crowd that we that we see online, great competition.

May may not be the most you know, great news for all the various possible participants, right?

You'd like to have fewer people to compete with, but for us that competition means we have a better probability of getting a really good solution for the warfighters.

And that's who we're really doing this all for.

Is to make sure that our command and control operators in the Air Force, but also the space force and beyond are able to leverage the know how and the ingenuity that you all have in order to fight and win our nation's wars and bring more Americans home.

Safely.

So I'm going to do my best to kind of give you a flavor of what you might experience there and what you what your software services would be expected to contribute to what we're doing.

And part of this is to motivate you to apply in the first place.

We understand it. It's not.

A trivial matter for your companies to put in the effort to submit for these things, so I'm hoping I can provide a little bit of motivation for you to continue with the process and compete to come to the event there a lot of fun.

So what I want to do is I just want to kind of paint a picture for you. First of all, we know that some folks have have perhaps been.

Into some of these ask me anything some folks have been to the dashes and hopefully some folks have never heard of any of this and we've got some new.

New companies coming in as well and essentially what is going to be occurring is on a, you know daily or even couple times a day we are going to have a crew of military command and control operators.

These are battle managers enlisted in officers that.

That are in charge of orchestrating the chaos of combat.

And they're gonna sit down in the facility that that Doug helps helps to run there. And they're gonna fight a battle for about an hour. And in the baseline case, they're gonna fight it with tools that we have, the representative tools that we have today.

That the war fighters are using and then in the experimental case, they're going to be using the tools you provide.

As you know, in the the information that you provide through the user interface that that Afro will be supplying and so the quality of your service will contribute to how they fight that hour long battle.

And we'll do that over and over and over. Some of the things.

That again, to entice you to keep going through the process and and to apply and and yes, not everyone gets selected, but we still.

Hope that you'll stay with us through this process.

Is that you get an opportunity to see first hand what these war fighters are experiencing in a in a very realistic.

Command and control combat scenario.

It's unclassified, but that doesn't mean that the chaos and complexity.

Is lacking at all.

We we throw the kitchen sink at the at the cruise and make sure that they have lots of dependent variables and and uncertainty and incomplete knowledge and incomplete information to have to work their way through just like our warfighters do in real combat scenarios.

OK, so and I'm hoping, just like Doctor Frost was saying.

As you have questions please.

Let's go ahead and ask them.

I'll try to keep track if if anybody has their hand raised, but you can always just come up on voice and interrupt me if you have a question for clarity. So.

In the.

In the hour long battle that they're going to be.

Fighting both, you know, without any help from machine partners that you all will be designing.

Or with it, there's going to be many things happening at once.

So let me paint a little picture here.

What you see in front of you?

We we set up our scenario.

Just kind of in a fake universe where the the bad guys are in the northwest, there on the peninsula and we've got contested territory over what in the real world is Cuba. And then forces coming in, you know, from the east and in this case I just want.

To direct your attention to.

The situation that's in the South, we're gonna. We're gonna focus on on that specific.

Situation. So we've got some F15 ES if you're if you're vis recy for particular.

You know, silhouettes is is not up to snuff.

No worries. That's all great experience that you'll get talking to the warfighters about, you know, real real aircraft, real missions, obviously at the unclass level. But we've got some F15 ES going after a mobile command and control unit, A classical approach in war fighting is take off.

The head of the snake so that the forces.

Then go into disarray.

So here's an opportunity to take out that mobile command and Enemy mobile command and Control unit.

We've got some F sixteens providing escort just in case those enemy fighters to the north suddenly decide to take interest in these activities, and we've got some F eighteens that are suppressing the surface to air missile system that you see in my little icon with a truck and.

A missile off there to the right.

And and so they're going after that target.

They received fuel from the tankers.

That you see out to the east and there's some other activity going on you see up in the Northeast as well?

B1 going after.

Maybe a mobile missile launcher?

So how did we get here?

How do we get to the point where this this mobile C2 unit?

We didn't know where it was before it emerged.

It popped up, started talking on its radios.

We were able to.

Perhaps figure out that.

It was.

It had moved locations and now we have an opportunity to target, but it's mobile, it could. It could move, it could escape before we can get there.

How did?

A battle manager cobble together.

The forces decide what needed to happen and get this small what we call a force package.

A strike package force package because we deal with multi domain.

Assets.

To the point where they.

It could be prosecuting that target and it's not just error assets. Here you see that BLOB.

That's kind of covering the area that's representative of satellite coverage.

How did they ensure that the satellite would be looking in the right area in order to characterize what that surface to air missile threat system was actually doing, where it was? Was it?

Is it a threat to?

What the F15 ES are doing?

Along with the rest of their force package.

How did we get here?

And so the the goal of Battleco production is to figure out all the various solutions that may or may not come to reality, that a battle manager can choose from in order to orchestrate.

Some some dynamic, complex and chaotic.

Mission or objective within the operational environment within the battle space.

So in order to have figured out, now put yourselves in the in the shoes of this battle manager, you are the battle manager.

How did you?

How did you know this was even an option?

Well, you had to have figured out that it was an option and you could say, well, well, didn't the planners come up with something and put it in the air tasking order?

Well, remember this was a dynamic target.

We didn't know where this particular target.

Was we may not even know it existed.

Maybe we didn't know that the adversary.

Had deployed their mobile C2 systems to to the island from their own territory yet.

So just like the quote says that you've you've all heard it. No plan survives first contact with the enemy as I used to teach my students, you know, no plan survives first contact with reality again. Whether other things can go wrong.

And sometimes the enemy doesn't even have to lift a finger for plans to to collapse.

And we've got to start.

Playing a pick up game.

OK so.

When we when we think about, well, how did we get here?

Well, maybe there were some options to either attack the C2, the mobile C2 node directly. In other words, just fly straight to it.

The option that we ended up using was to do an end around the one labeled #3 to go around the threat and maybe there was another option to go to the north, but that ended up you know there was no.

Good way to to do that and mitigate all the risks.

Well, how do we know we had those three? Really two options. We considered the 3rd and rejected it.

And what's the difference between them that that blue path takes longer?

To to traverse that means this mobile unit could pack up and and skedaddle before we have an opportunity to target it, which means it can come back and keep fighting.

In the future, maybe the direct route we get there quicker, but we face more risk at some point.

You, as a battle manager, would have had to choose between these options, not to mention everything else going on in the operational environment.

How how can we?

How can we do that?

How can you, as an putting yourself in those shoes at battle Lancer?

How do you know what your options are?

And that that.

Statement right there.

Kind of the crux of the battle.

Co production ensemble. The combination of perceive actionable entities, match effectors, and sequence battle clause. Those names of those functions. Don't worry too much about trying to, you know, tease out exactly what the words are.

They're just reminders for us about what this decision what these three decision functions are all about.

It's more important for you all to understand the spirit if you will.

The the the reality of of what those decisions are, what those things that humans right now have to do with their own brains, that they could be teeming with, with technology, with, with machine partners, to do better, faster and at greater capacity.

So the the one of the questions that that one of the first questions that somebody asked earlier was you know, how can we, hey, we're a new company.

How can we be competitive?

Well, one of the main ways to be competitive here is to prove to us in your submission that you understand.

These decision functions that you understand the battle Coe production ensemble. What you're really trying to get after and what you are really trying to get after is how well.

A software service can you know, team with humans to explore and understand the solution space.

OK, I know there's some technical jargon in there, but I think most of you are technical folks, but you can imagine that with a dynamic battle, a lot of things moving information is not complete.

You don't have perfect information about the the operational environment. In fact, you have very scarce and limited information about the operational environment.

How do you even know what what the solution space looks like?

And the the ensemble, the cycle through PAEMEFN GB C is a a model based systems engineering decomposition of what it takes to actually understand that solution space. And when I say solution space think of a solution as battle Coas. What are.

All the battle Coas that are available.

And here we just looked at, for instance, the option to strike that mobile C2 unit with the F15 ES.

But there were probably other options to use the B1 or the F30 fives or one of the ships.

All those potential options are part of that solution space.

Where those battle codes are and what we are looking for is to see.

Do you understand that your role is to explore that full solution space as fully as possible, and then understand it?

Because we're not.

While we do want to see all the battle Cos that your software service comes up with, we are going to force you to pick three. Put your company's name behind them and say, hey warfighter, we think these are the the best 3 for this particular problem or.

Target.

So it's that exploration and understanding.

Of the solution space. If you cross out solution put in battle KOA space all the potential battle codes that are out there. We need machine partners to help us explore because as humans, how many of those solutions can we really explore in the time that we have which?

May be seconds to minutes in order to really think about what are all the implications.

How would they come about all the details that need to go in there for for most operators?

You may think of one or two at best, and then you just have to go do one gut check to say, OK, the option.

I just thought of isn't isn't terrible or stupid.

All right, we're going to go with that and then figure things out as as the as the battle unfolds.

If instead we had machine partners who could say, hey, in addition to the idea that just popped into your human head, here's three to five other.

That you could apply to this particular target or this problem because it's not always a target you want to strike.

It might be a shipping friendly ship in distress.

It might be resupply mission.

It might be some other kind of problem to address.

Here's here's three to five options, and here's why. You know they they will work and how they compare to each other. Having that immediately available.

Instead of having to build all those. Now the cognitive work for the.

For the human operators, shifts to you know, either integrating, which hopefully will get machine partners.

That's a different decision function and then ultimately choosing those options.

Instead of only exploring a limited part of that solution space and being.

Essentially subject to that limitation, that's that's the only idea you have.

So it's what you have to go with.

We're trying to break free of that and you all are going to help us do that.

So like I said, the transformational model for decision advantage is a model based systems engineering approach to get inside the head of warfighters.

We are for the command and control and effects at scale project within.

Afrl we are focused on command and control and ISR.

So what?

We break those down into four, what we call decision domains of battle planning.

Battle Command, Battle management and battle analysis.

Think of battle analysis like military intelligence and surveillance.

And those four domains get after four fundamental questions.

Command is what you know is is what ascribes value to any particular action or goal or strategy that we might have.

Battle analysis helps us try to take that tiny bit of of uncertain information.

About reality that we have and figure out what what is actually going on or what, what what is likely going on planning is is what looks into the future and says well, what if if we had certain conditions what could we do and battle management where we're focus.

Is looking at the battle as it is unfolding.

We don't get to choose like the planner does.

Various what ifs were in the what if? If you will and we have to.

Use. Take that information about what the commander has said matters that we accomplished and what analysis has told us about what is real and figure what should we do right now.

Should we even go after that mobile C2 unit?

Maybe there are other more lucrative targets, or maybe there are more urgent situations that require our limited resources on the right. You can see those.

13 functions that are on the on the cheese wheel on the on the left there.

Battle management.

The three highlighted that we are focusing on PAEMEF and GBC. You can see them arrayed in a slightly different depiction there, just in terms of some dominant information flows between the various functions of battle management and on the right, you see how PA.

Flows from.

With a entity of interest flowing into PAE and a battle effect effect flowing out into match effectors.

That match effectors takes that battle effect, IE some action in the operational environment and produces what we call an effect effector match basically gives resources as assets for a particular task, and then generate battle. Cause takes that and builds a force package around what we call the prin.

Effect effector match or the main if you will in in a in a traditional strike pack, the main striker who is going to attack primarily attacked a target. In our case that principle effective vector match, one of them was the OR is the F15 E's it.

Could have also been the B1. It could have also been the F.

30 fives and we're going to focus kind of on that one example.

To.

To understand how these things all kind of interact.

So really important though, you'll notice that there's a little dotted line going from GBC back up to PAE.

This is the the realization of the ensemble and one of the questions that y'all one of you asked before you know, do you care about the quality of the individual decision functions?

Or their interactions and and just like Doctor Frost said.

We care about you being able to perform the logic of of each of these decision functions, but the overall quality of the output of your battle cause will be reflected.

IE the interaction of these will be reflected in their individual quality. So to get a good GPC out.

We we surmise that somewhere in your code you know, PAE had to have happened and Nef had to have happened.

Now whether your code.

Modular arises.

The you know the the ensemble like we have here or not.

That's up to you.

The the model is a the transformational model for decision advantage is a government reference model.

That means you can use it to design solutions, but you don't have to have.

The caveat though, is if you don't use this model to design your solutions, whatever you do, design.

Has to perform as well or better than anything that would be designed against this.

In other words, it's you've got to be able to show your homework that you're actually…