Magnus - a simplified workflow definition language which works in local to Argo without any change
[Link to the repo](https://github.com/AstraZeneca/magnus-core)
Magnus is a simplified workflow definition language that helps in:
* **Streamlined Design Process:** Magnus enables users to efficiently plan their pipelines with [stubbed nodes](https://astrazeneca.github.io/magnus-core/concepts/stub), along with offering support for various structures such as [tasks](https://astrazeneca.github.io/magnus-core/concepts/task), [parallel branches](https://astrazeneca.github.io/magnus-core/concepts/parallel), and [loops or map branches](https://astrazeneca.github.io/magnus-core/concepts/map) in both [yaml](https://astrazeneca.github.io/magnus-core/concepts/pipeline) or a [python SDK](https://astrazeneca.github.io/magnus-core/sdk) for maximum flexibility.
* **Incremental Development:** Build your pipeline piece by piece with Magnus, which allows for the implementation of tasks as [python functions](https://astrazeneca.github.io/magnus-core/concepts/task/#python_functions), [notebooks](https://astrazeneca.github.io/magnus-core/concepts/task/#notebooks), or [shell scripts](https://astrazeneca.github.io/magnus-core/concepts/task/#shell), adapting to the developer's preferred tools and methods.
* **Robust Testing:** Ensure your pipeline performs as expected with the ability to test using sampled data. Magnus also provides the capability to [mock and patch tasks](https://astrazeneca.github.io/magnus-core/configurations/executors/mocked) for thorough evaluation before full-scale deployment.
* **Seamless Deployment:** Transition from the development stage to production with ease. Magnus simplifies the process by requiring [only configuration changes](https://astrazeneca.github.io/magnus-core/configurations/overview) to adapt to different environments, including support for [argo workflows](https://astrazeneca.github.io/magnus-core/configurations/executors/argo).
* **Efficient Debugging:** Quickly identify and resolve issues in pipeline execution with Magnus's local debugging features. Retrieve data from failed tasks and [retry failures](https://astrazeneca.github.io/magnus-core/concepts/run-log/#retrying_failures) using your chosen debugging tools to maintain a smooth development experience.
Along with the developer friendly features, magnus also acts as an interface to production grade concepts such as [data
/r/Python
https://redd.it/1ar66d5
[Link to the repo](https://github.com/AstraZeneca/magnus-core)
Magnus is a simplified workflow definition language that helps in:
* **Streamlined Design Process:** Magnus enables users to efficiently plan their pipelines with [stubbed nodes](https://astrazeneca.github.io/magnus-core/concepts/stub), along with offering support for various structures such as [tasks](https://astrazeneca.github.io/magnus-core/concepts/task), [parallel branches](https://astrazeneca.github.io/magnus-core/concepts/parallel), and [loops or map branches](https://astrazeneca.github.io/magnus-core/concepts/map) in both [yaml](https://astrazeneca.github.io/magnus-core/concepts/pipeline) or a [python SDK](https://astrazeneca.github.io/magnus-core/sdk) for maximum flexibility.
* **Incremental Development:** Build your pipeline piece by piece with Magnus, which allows for the implementation of tasks as [python functions](https://astrazeneca.github.io/magnus-core/concepts/task/#python_functions), [notebooks](https://astrazeneca.github.io/magnus-core/concepts/task/#notebooks), or [shell scripts](https://astrazeneca.github.io/magnus-core/concepts/task/#shell), adapting to the developer's preferred tools and methods.
* **Robust Testing:** Ensure your pipeline performs as expected with the ability to test using sampled data. Magnus also provides the capability to [mock and patch tasks](https://astrazeneca.github.io/magnus-core/configurations/executors/mocked) for thorough evaluation before full-scale deployment.
* **Seamless Deployment:** Transition from the development stage to production with ease. Magnus simplifies the process by requiring [only configuration changes](https://astrazeneca.github.io/magnus-core/configurations/overview) to adapt to different environments, including support for [argo workflows](https://astrazeneca.github.io/magnus-core/configurations/executors/argo).
* **Efficient Debugging:** Quickly identify and resolve issues in pipeline execution with Magnus's local debugging features. Retrieve data from failed tasks and [retry failures](https://astrazeneca.github.io/magnus-core/concepts/run-log/#retrying_failures) using your chosen debugging tools to maintain a smooth development experience.
Along with the developer friendly features, magnus also acts as an interface to production grade concepts such as [data
/r/Python
https://redd.it/1ar66d5
GitHub
GitHub - AstraZeneca/magnus-core: Core package of magnus
Core package of magnus. Contribute to AstraZeneca/magnus-core development by creating an account on GitHub.
I created a video on why you should be careful when using Python dictionaries as function parameter
This talks about mutability as Changes inside a function affect the original dictionary which could lead to unexpected behaviors and hard to debug issues.
Here is a link to the video
https://www.youtube.com/watch?v=zTTDQePffxU
/r/Python
https://redd.it/1d9upsj
This talks about mutability as Changes inside a function affect the original dictionary which could lead to unexpected behaviors and hard to debug issues.
Here is a link to the video
https://www.youtube.com/watch?v=zTTDQePffxU
/r/Python
https://redd.it/1d9upsj
YouTube
Avoid Using Python Dictionaries as Default Parameter in Functions! Here's Why...
Are you using Python dictionaries as default parameters? Avoid it! 🚫 This common mistake can lead to unexpected bugs and hard-to-trace errors in your code. In this video, I break down the dangers of using mutable default arguments, especially dictionaries…