58b20109cf
import sys
import yaml
with open(sys.argv[1]) as fp:
data = fp.read()
if not data.find("---") == 0:
# no head
print("NO YAML HEAD FOUND")
sys.exit(-1)
data = data[3:]
head_end = data.find("---")
head = data[0:head_end]
data = data[head_end+3:]
metadata = yaml.safe_load(head)
cats = metadata.pop('categories', None)
if cats != None:
if type(cats) == list:
tags = cats
elif type(cats) == str:
tags = cats.split()
tags = list(map(lambda t: t.lower(), tags))
metadata["tags"] = ", ".join(tags)
new_data = f"---\n{yaml.dump(metadata, default_flow_style=False)}---{data}"
# write it
print(f"coverted: categories to tags: {tags} - {sys.argv[1]}")
with open(sys.argv[1], "w") as fp:
fp.write(new_data)
sys.exit(0)
if not metadata.get("tags", None):
metadata["tags"] = "untagged"
new_data = f"---\n{yaml.dump(metadata, default_flow_style=False)}---{data}"
print(f"untagged: {sys.argv[1]}")
# write it
with open(sys.argv[1], "w") as fp:
fp.write(new_data)
sys.exit(0)
print("No changes needed")
77 lines
3.5 KiB
Markdown
77 lines
3.5 KiB
Markdown
---
|
|
layout: post
|
|
title: "Experiments In Owning Data: Part 2"
|
|
date: 2019-01-23
|
|
comments: true
|
|
tags: freebsd, nextcloud, data
|
|
---
|
|
|
|
_[Checkout part 1](/blog/2019/01/20/experiments-in-owning-data/index.html)_
|
|
|
|
I wanted the setup to have as much less friction as possible, more time
|
|
maintaining systems and uploading data is less time for things that I actually
|
|
enjoy. Luckily a lot of open source tooling exists to help in this space, some
|
|
mature, some not so much.
|
|
|
|
I stumbled upon these two apps during this experiment and they have become my
|
|
daily drivers of sort -
|
|
|
|
### Nextcloud
|
|
|
|
[Nextcloud](https://nextcloud.com/) is your one-stop cloud storage app. The core
|
|
of nextcloud is a file storage server with a web interface, easy enough right?
|
|
True to the open source nature it is heavily [extendable with apps (more like
|
|
plugins)](https://apps.nextcloud.com/) - and there is an app for everything!
|
|
hosting calendars, editing office docs, taking markdown notes and even video
|
|
calling.
|
|
|
|
Nextcloud comes with companion mobile apps for Android and iOS. The apps let you
|
|
access and upload data to the nextcloud server. They can also automatically
|
|
backup new photos to nextcloud! Loosing a device never means loosing photos.
|
|
|
|
The server also exposes a [webdav](https://en.wikipedia.org/wiki/WebDAV)
|
|
interface, so one can just mount it as a folder with
|
|
[davfs](https://savannah.nongnu.org/projects/davfs2) over fuse. Webdav is not
|
|
known for being very performant, but I like the simplicity of it and works well for my day today needs where I just need to work on a couple of small files.
|
|
|
|
Nextcloud has replaced Google Drive/Photos for me. The whole process is not
|
|
without pains. Documentation exists and is helpful, but sometimes falls short.
|
|
I am yet to figure out how to setup
|
|
[Collabora](https://nextcloud.com/collaboraonline/) (online office suite).
|
|
|
|
### Tiddlywiki
|
|
|
|
[Tiddlywiki](https://tiddlywiki.com/) claims itself to be a non-linear personal
|
|
notebook. That's quite vague, because it depends entierly on the user. I use it
|
|
to store tiny bits of notes, lists, code etc. I have managed to move everything
|
|
from evernote to tiddlywiki with the exporter plugin. Formatting of exported
|
|
notes is not great, but the content is fine - and that's all I care.
|
|
|
|
|
|
## Current and Future
|
|
I have been running this for almost an year now, other than nextcloud and
|
|
tiddlywiki, the server also runs an email server (not my primary one yet), code
|
|
repo and a CI setup.
|
|
|
|
Where to now? I am obviously missing out on features, and I had to spent more
|
|
time setting it up than signing up for a service. But the setup is tailored to
|
|
my use and so far I am happy with it.
|
|
|
|
That is not to say I think its perfect - I have a wishlist!
|
|
|
|
### 1. Better photos interface and search
|
|
I miss the fast and accurate photos search on google photos, and the current
|
|
nextcloud photos interface is slow and only has file system hierarchy as a
|
|
grouping mechanism. Maybe I can [inception-v3](https://arxiv.org/abs/1512.00567)
|
|
or any other similiar image classification models and build an auto tagging
|
|
system?
|
|
|
|
### 2. Disaster proof backups
|
|
Current backup system is quite rudimentary, it copys data every day to another
|
|
server in the same datacenter (also on `raid1`). Its not great if the DC itself goes
|
|
under - I would like something offsite, and also to backup server configs.
|
|
|
|
Well, that's about it for now. :-) I do have a question to the reader: do you
|
|
know any applications in this space that I should know about? please let me know over
|
|
[twitter](https://twitter.com/notmycommit) or [email](mailto:mail@dbalan.in).
|