TaRDIS-APIs/Artificial-Intelligence-and-Machine-Learning-APIs.md

@@ -521,7 +521,13 @@ The current algorithm considers the following configuration options:
 
 **save_interval** - Interval (in steps or episodes) at which the model or
 
-### 6\. Integration with Babel and deployment in real networks
+## FAuNO Standalone
+
+We continued the development of the FAuNO framework and we are integrating FAuNO to extend the babel framework and provide decentralized orchestration for task processing networks of nodes, we call this component of the project as FAuNO Standalone (FAuNO S), and the participating nodes as FAuNO Standalone Nodes (FSn). The idea of FAuNO S is to fullfill the capabilities mentioned in the deployment diagram in figure 1.
+
+![FAuNODeployment](uploads/994bf18face4c0cd541f757ba92af364/FAuNODeployment.png)
+
+Fig. 1: Diagram detailing the types of FAuNO diagrams available.
 
 For the code and more information on the integration with babel see the [v0.1 of the solution](https://github.com/FredericoMetelo/FAuNOStandalone.git)
 
@@ -534,6 +540,104 @@ For example by cloning with:
 git clone https://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@github.com/FredericoMetelo/FAuNOStandalone.git
 ```
 
+### FAuNO FRL Algorithm
+
+The core design of the FAuNO Orchestration component remains the same. To recap from D5.2 FAuNO's orchestration follows a standard Client/Server Federated setup, divided into local and global training components. Local training uses Proximal Policy Optimization (PPO)[1], an Actor-Critic reinforcement learning algorithm. Both actor and critic are implemented via deep function approximators. For global training, we adapt the FedBuff [2] approach, using a buffered asynchronous scheme where the server aggregates the first *K* updates rather than waiting for all participants. Our extension supports continuous control: agents continuously train and send updates, replacing previous ones and applying weighted aggregation. Once sufficient updates are collected, a FedAvg-style aggregation updates the global model, which is then distributed to all participants.
+
+We continued development based on the FAuNO results, culminating in a paper currently under submission to NeurIPS. The testing workflow was streamlined, with mechanisms added to support rapid experimental iteration. We further fine-tuned the model and improved performance. Lastly we implemented the state-of-the-art algorithm for task offloading in edge systems SCOF [3] and benchmarked FAuNO against it.
+
+
+### Architecture of FauNO S
+
+There are a total of three main modules to the FAuNO Standalone Framework:
+
+- Task Processing Module - Responsible for processing the tasks and management of the task queue.
+- Event Handling Module - Responsible for linking of the different events with the appropriate handler. There a total of four handlers
+- RL Orchestration Module + Federated Manager Module
+
+
+An overview of the components can be observed in the fig.2 image. In the subsequent sections we will go over what each of this components entails.
+
+![FSnComponents](uploads/a4fee2ec44e15f74d9f7de089709be56/FSnComponents.png)
+
+Fig. 2: Diagram containing the compenents of an FSn node
+
+####  Task Processing Module
+
+![FSnTaskProcessingModule](uploads/081afc9b5e0bb3baa2690b75ca252a3c/FSnTaskProcessingModule.png)
+
+
+Fig. 3: Task processing module components
+
+The task processing module is responsible for concluding the task assigned to a given FSn. The Task processing module has three sub-components:
+
+Task Queue: Stores the tasks that need to be processed, tasks are stored in arrival order.
+
+Task Manager: Pulls the tasks from the queue and assigns them to a processing unit. It handles the task conclusion process and the selection of the next task to be offloaded considering that some tasks were assigned for local processing. This is the same mechanism that when prompted selects the next available task for the offloading decision
+
+Task Processing: The component responsible for processing the tasks. This component is still under refinement and we are still considering other approaches, but currently the task processing is handled by having the clients pass executor mechanisms to the FSn, that can be used to execute any one of the required processing dynamics that the client may need.
+
+#### Event Managing Module:
+This is the component that will orchestrate the flow of the computations. At the core of the event management component is the event queue and the event processing engine, which focuses on pulling events from the event queue and calling the appropriate processing mechanism. We can allow the addition of event-processing mechanism pairs to extend the functionality of the framework as needed. The initial predicted events are generated by the client, neighboring nodes, and the internal FSn working mechanisms. 
+
+![FSnEventMan](uploads/364675c4154e75478c76683795244a80/FSnEventMan.png)
+
+Fig. 4: FSn Event Manager component
+
+**Client Events**
+
+We predict that there will be two types of events that involve the clients directly. The first is the generation of new applications that need to be processed. And the second is the sending of results from apps that reached a terminal state to the client that created the app.
+
+1. App Generation Event: When a client creates an app in the form of a graph of tasks these need to be registered in the node and unrolled into the executable tasks. So the event handler for the application unrolling needs to be able to maintain a data structure tracking the progress of the apps. 
+
+2. Sending Results Event: This should be part of the joint responsibilities of a task-finishing handler and the application manager. The event that pertains to the client directly is the app conclusion, where the results are sent to the client. The other events will be elaborated on in the task events handling.
+
+**Task Events Module:** 
+
+The task events module is meant to manage the conversion of apps into tasks after their registering and pre-processing, and their continued unrolling into more tasks as progress is made. The main concern of this module in reality is the handling of task completions and the handling of offloaded tasks. When a task is completed one of three events is created.
+
+1. Response of an offloaded task: An offloaded tasks needs to have it’s results sent to the node responsible for the app that generated said task.
+
+2. Conclusion of task: Is composed of two sub-events, the updating of the progress in the app, and the addition of the remaining tasks that can be processed. If the task is completed, then the results need to be sent to the original client.
+
+2.1 Unrolling of app: Progress needs to be tracked, and the tasks unlocked by the task that concluded need to be added to the task queue.
+
+2.2 App finish: If a task concludes an app the appropriate event needs to be handled.
+
+3. Task offloaded to node conclusion: The original node responsible for task needs be sent the results. 
+
+4. Receive results of an offloaded task: Have a similar handling to the local task conclusions.
+
+5. Task Offload Event: The node prepares and shares with a neighbor one of the tasks in it’s task queue.
+
+**FSn Events:**
+
+The FSn events are related to communication and information exchange between nodes in the FSn network. Its main events are collecting local state information, scheduling broadcasts, and processing incoming state data from neighbors.
+
+1. Receive Neighbor State: Handles incoming local state information from neighboring nodes, updating internal representations accordingly.
+
+**Federated Updates Events:**
+
+Depending on the type of node it may receive one or two types of FL updates. All nodes receive the latest global model. And if the node is the global model manager, it will receive the updates from all the other nodes.
+
+1. Global Model Update: FSn receives the latest global model to update it’s own local version.
+2. Local Model Update: The FSn responsible for managing the global model must receive the local model updates sent by the other FSns and handle them accordingly.
+
+#### Orchestration Module
+
+![FSnMO](uploads/b963a4ef977d6fc49b605ca0a9597146/FSnMO.png)
+
+
+Fig. 5: Orchestration module components
+
+The orchestration module is responsible for the offloading decision-making and management of the FRL flow. The OM will continuously decide for each task (obtained by requesting tasks to the task management module) in the queue whether to offload it or to process it locally, based on the available information maintained by the module. If a task is to be processed locally, this information is passed to the task management module. If the task is to be offloaded an event is created and a handler that can prepare the task for offloading and send it through the appropriate API is called.
+
+The OM is also responsible for handling the RL training loop, using a delayed feedback technique to acquire the necessary information for the training of the network. 
+
+The last responsibility of the OM is the handling of the federated training. There are two types of nodes, the regular FSn and the global manager FSn. The regular FSn periodically after any given number of rounds of training sends the weights to the node currently managing the federation, the global manager FSn. The global manager FSn in turn is responsible for receiving the updates sent by the other nodes and storing them to aggregate into a new global model. An election mechanism in case of failure of the global manager also needs to be set in place.
+
+
+
 **References**
 
 [1] Alberto Montresor and Márk Jelasity. "PeerSim: A Scalable P2P Simulator." Proceedings of the 9th International Conference on Peer-to-Peer, 2009, pp. 99-100.