Author(s: Shubham Sboo Data Science Data Scientists have the right to view, preview, share and fork data models and code. DAGsHub is a collaborative Data Science Platform What’s DAGsHub? DagsHub, an open-source platform for data science and machine learning, allows you to rapidly build and scale machine learning projects using the power of DVC (Data Version Control) and git (Source Code Versioning). DAGsHub brings together data, code, and mode – all in one spot! The data professionals have been struggling to handle code and data since the beginning of this field. Unlike traditional software engineering projects, where only the code is tracked, ML projects require you to also track data and models. This can be a difficult task. You can relate to all the components, including code, data and monitoring, that are involved in an enterprise-grade, ML project. It’s a daunting task trying to combine all these pieces into a cohesive whole. Standard code versions platforms such as GitLab, Bitbucket or GitLab don’t allow for pushing large amounts of data. Traditional Solution: To manage code and data, push code to any code versioning platform such as GitHub. Then push data and models to the on-prem storage or cloud storage, like AWS and Google Cloud. The first problem with having your code and data stored in different locations is the link or bridge. Your ML project will function correctly if all the threads are efficient. Another problem you might face is latency, which can affect the speed of your application. DAGsHub Storage — A Way Forward DVC (Data Version Control), which allows you to manage your data in a similar way to how git handles the code, was invented. The era of efficient tools and methods for managing ML projects from start to finish has arrived. DAGsHub Storage, built upon DVC, brings together the essence data science, i.e. data and code. The DVC remote requires no configuration, and it works right out of the box. Sharing data and models is as simple as sharing a hyperlink. This allows for easy collaboration between data teams and encourages free flow of ideas. DAGsHub Storage makes it easy to track and compare versions of version data, models and code like git. DAGsHub’s repository interface provides an automatic pipeline that allows everyone to see the parts of the project as well as how they connect together. This makes it possible for all members of the team to easily understand the work flow of the project, regardless of technical knowledge. You can also compare two data types side-by-side. It supports text, audio and tables. Side-by-Side Image data comparison Where DAGsHub Shines? DAGsHub makes it easy to share, reuse and build machine learning projects and data science projects quickly. This eliminates the need for data teams to create every project from scratch. DAGsHub is unique because it has built-in remote tools such as Git (for source codes tracking), DVC(for data version tracking), MLflow (for experiment track tracking) which allow you to link everything together in one location with no configuration. DAGsHub makes it easy to monitor and track the various ML experiments of different people using a user-friendly interface. You can track all the experiments in a ML project and link them to their specific versions of code and data. The Experiment Tracking Dashboard You can track your experiments and also see the differences in hyperparameters and performance. This is done using the experiment’s recorded values and the intuitive visualizations that DAGsHub provides. Interactive visualizations for experiment comparison Platforms like DAGsHub are set to become more mainstream. They will be able to execute collaborative data projects across organizations and sectors. This will allow data teams and data scientists to collaborate, share, and quickly create their machine learning and data science projects. References https://dagshub.com/docs https://dagshub.com/blog https://dagshub.com/Shubhamsaboo/summarization If you would like to learn more or want to me write more on this subject, feel free to reach out. Twitter DagsHub – Github for Data Science originally appeared in on Medium. People are responding and highlighting this story. Published via
