Testing and Infrastructure

On recently passed DevOpsStage where we had discussion about tests and testing in Brave New Infra World

Record here (Ukrainian and Russian)

Links on mentioned tools and configs:
- pre-commit - framework for multi-language pre-commit hooks
- pre-commit-terraform hooks by @antonbabenko
- pre-commit-config - list of Infrastructure useful hooks and it dependencies by @MaxymVlasov
- Terratest - testing of infrastructure code
- Molecule - testing of Ansible roles

#testing #terraform #git #culture

It can be used as template in all cases when you spend more time debugging something than fixing it.

#git #culture

​​Git Command Explorer позволяет находить команды Git по простым запросам типа “I want to remove files from old commits”

#git

​​Work with DBs as simple as code in git!

If you missed, git 2.26 released 3 weeks ago, where set protocol version 2 to default, add some new features and improvements to stuff like git grep - this definitely described in release highlights.

But not, git still can't be used as versioning/sharing stuff for databases. Dolt can.

Dolt is Git for data. Instead of versioning files, Dolt versions tables.
It is inspired by RDBMS and Git, and attempts to blend concepts about both in a manner that allows users to better manage, distribute, and collaborate on, data. And it have own Hub, with blackjack and databases!

Here some use case how it can be used:
- Sharing Data on the Internet
- Ingesting Data You Do Not Own
- Versioning Data Lake Query Output
- Reproducing Models or Analysis
- Better Database Backups
- Add Data Versioning to an Application

#git #database

​​Happy Friday!

#git #github

Обновите свой git, пжалста

https://www.openwall.com/lists/oss-security/2021/03/09/3

#git #exploit

git-xargs - tool for change same things in many Github repos at once

This is a CLI tool for easily executing commands and scripts opensorced by Gruntwork a short time ago.
Written in Go and used goroutines, so it pretty fast, except you reached the Github API call limit :)

#git #github #toolz

"When I see a door with a push sign, I pull first to avoid conflicts" - anonymous

In version 2.23 of git, two new commands have been introduced: git switch and git restore. Both are aimed to replace ambiguous behavior of git checkout. Although, the original checkout command is still available.

Switch works the same way as checkout on branches and restore as checkout on files.

Why? Because the previous implementation was confusing.

#git

Our tech stack differs from one company to another. However, there are certain things that almost everybody use. Like, for example, Git!

Here are some release notes for Git 2.34

This release introduces the use of sparse index for some of git commands.

You can read more about sparse checkout and sparse index here.

This is especially useful for monorepo users. Although, I haven't being working with one for more than 2 years now, I have some repos in mind, where I would like to test it.

As a bonus: An article about Git's data structures and their behavior. Commits are not diffs, folks!

#git

​​Let's continue talking about git.

First of all, if you lack of understanding how git works - three-git-tips will help you.

If you are already a "pro user", check out git-extras, it commands can be helpful.

#git

​​Git is simply too hard.

I have mixed feelings about this article. On one hand, Git is too hard indeed. I personally google certain Git operations from time to time.

On another hand, this article doesn't provide any alternatives or ideas on how to fix this situation. Also, a lot of Git complexity comes from its distributed nature. Distributed systems are much harder to architect compared to centralized ones. Although, currently, we're mostly using Git as a centralized system nowadays.

So, I'll just add this well-known comics here.

#git

It may happen that you have a catch-all repository to store some work that may or may not be important. Notice, it's not quite a monorepo pattern, rather a single junkrepo. Like, I saw devops, utils, tools, and similar repositories in quite a few companies.

But what if some things from those repos are actually important, and you want to have a dedicated home for those things? Here's a guide by GitHub that explains how to separate things from a repository while saving the Git history.

This guide is created by GitHub, but it should work with any Git repository. Although, I only tested it with GitHub.

Enjoy!

#git #github

​​git branches: intuition & reality - an explainer by Julia Evans.

#git

An opinionated article that claims that one has to keep all the dependencies in version control.

Now, I don’t have strong opinions here. On one hand the proposal in this article sounds extreme. On another hand, my team has quite a few Go projects and we use vendored modules there.

So, we actually keep the dependencies in VCS. As anything else, it has its pros and cons. I could build a project and run some isolated tests on a plane. But also it almost impossible to review a PR that touches that vendor/ directory.

The author points out that it’s Git’s fault that it’s not designed to work with large number of files and files of a large size. Yes, I’d ask for a reality check here, some Git is omnipresent these days.

#programming #git #opinion

I know that many folks have a mix of corporate and private repositories on their laptops. By private, I mean their small projects, dotfiles repo, forks of public repositories, etc.

Here's an interesting guide on one of the ways of how to keep separate users (email, signing key, name) for different repositories.

This would also work if you have repositories backed by different provides. For example, if you need to push to both GitHub and GitLab and use different SSH keys for that.

Here's a condensed version of this article on StackOverflow.

#git

After reading this good retrospective article, I can say for sure:

#git is fine.

Many years ago there was an open lecture or rather Q&A in my university by one of Facebook engineers.

He mentioned that they do not use Git, to which I was surprised and asked why. He didn’t provide a concrete answer, since it was like this when he joined.

However, this article may have an answer to why Facebook uses Mercurial instead of Git. “May” because obviously there’s just one viewpoint.

Tl;dr: because they had a unique use case as well as unique amounts of money to contribute to a custom solution. Still, it’s interesting that their initial proposal to contribute to Git was sorta declined.

So, here we are. Does it mean that you also need to switch to Mercurial to be like Facebook? No! Very likely you’re not Facebook. Moreover, I only used 1 project stored in Mercurial in my life (I’m not trying to argue that there are more).

Still, I like such articles on the topic of “tech history”. I think they help us to better understand why we are where we are and how did we get here.

#git #hg #facebook #history

Anyone can Access Deleted and Private Repository Data on GitHub

Sounds scary, if you didn't deep dive into how git works and how GitHub hosts it.

TL;DR: If some repo can git fetch upstream - all these fetchable commits will be always accessible from this fork/main repo.
Only after the visibility of the repo changes - new commits will be not discoverable.

For more details and examples, check this article, which was brought to us from CatOps Chat.

#security #git #github

Git is one of those things, where you can discover new things even after decades of using it.

This article provides some tricks you could do with Git. If you knew those already - that's cool. If not, you're welcome :)

#git

Recently, Git celebrated its 20th anniversary, and the pre-commit framework marked 11 years.

Interestingly, despite their longevity, these technologies are still not being used to their full potential — especially when it comes to leveraging git hooks.

To shed some light on this, I gave a talk about them (in Ukrainian): https://youtu.be/mqzyVg5WOMM
This is a natural continuation of my previous talk on the same topic, which I gave back in 2018 :)
Covered (and skipped) topics and link to the older talk in video description. Enjoy!


#git