Posted on

DNS Records and Services

First, there are two kinds of DNS records: those for client look, and those for a server.

Client Lookup

I don't trust Google DNS, though for a while it was the go to DNS, and easy to remember at 4.4.8.8 8.8.4.4 and 8.8.8.8. For privacy, for me, there are two options, with the first being just better: - dns.watch 84.200.69.80 / 84.200.70.40 - 1.1.1.1 / 1.0.0.1 If one wants some security (as a service), then Quad9 is worth a look.

DNS Services

There are several DNS services to choose from. Dyn and related companies is the worst. Free DNS services such as afraid.org and he.net are unreliable, or simply not reliably fast. It makes the most sense to go with a top-rated DNS service (highly available and fast resolve times), and pay for this service (though less is more when it comes to expenses). - DNSmadeEasy.com - Silly name, $30/year for 10 domains, fast and reliable. Generally in the top 10 of private resolvers. I've not found better/faster for cheaper.

DNS Records

NS Records

There are several records to worry about. The first are nameservers, which are put into the registrar database. This can be as few as two or as many as six (possibly more).

A Records

Depending on the DNS Server, these can have wildcards or not. Generally there are at least three A records to have: - Root domain - www subdomain - * wildcard For certain services, it is required to have a www. and also people mistype this, so it is best to have it as a domain, to have it on the SSL certificate, and to have a reroute from www. to the root domain.

CNAME Records

Usually only Bing Webmaster Tools requires a CNAME record. Otherwise these are generally worthless.

MX Records

These are for the mailserver. Usually a few are needed, one plus two backups. Gsuite has five records, but that is overkill. The top three make the most sense. Also, there are priority numbers, e.g, 1, 5, 10 to govern the round robbin-style resolving. - 1, aspmx.l.google.com. - 5, alt1.aspmx.l.google.com. - 5, alt2.aspmx.l.google.com.

TXT Records

TXT records are the go to place for every third party to put their info. Several examples of TXT Records include: - Yandex Webmaster Tools validation - Google Webmaster Tools/Analytics/GSuite/etc. validation - _acme-challenge records for DNS-based authentication for LetsEncrypt

PTR Records

PTR records are essentially a reverse so that an IP address is associated with a host.domain.tld. This is key for sending email.

DKIM, SPF, DMARC

These are all records for email security, at various levels. DKIM and DMARC are TXT records, and SPF can be TXT or specific SPF records, depending on the DNS service provider. - Setting up Gsuite DKIM, SPF, DMARC - Google on DMARC records - Test SPF and DKIM - Google on SPF - DKIM on Gsuite - Google: About DKIM

SPF Records

SPF looks like:

host.domain.com / "v=spf1 include:_spf.google.com ~all"

SPF are one of the earliest and easiest email records to set up for security, and specifically states which hosts can send email for the domain.

CAA Records

These records help tell SSL Cert providers which of those providers can generate a cert for the domain records. Each host needs two records: - Name (host), Type: iodef, Value: "mailto:address@domain.com" - Name (host), Type: issue, Value: "letsencrypt.org"

Posted on

PHP and MariaDB on Debian

Note: instructions for installing and configuring phpMyAdmin also included below.

Related Artices in Debian Services and Applications - Debian on AWS Lightsail - OpenVPN on Debian + UFW Firewall - Nginx and Letsencrypt on Debian - PHP & MariaDB on Debian

- Grav CMS on Debian

As of December, 2018 there are decent performance gains with the latest PHP and MySQL (MariaDB, not Oracle) versions. These are: - PHP 7.3.0 released 06 Dec 2018 - MariaDB 10.3.11 released 20 Nov 2018 PHP 7.3 outperforms PHP 7.2 and earlier versions on nearly all real-world web cms platforms. At the same time, MariaDB does indeed have performance enhancements which generally make it faster than the Oracle offering. For MariaDB the performance advantages have been apparent since at least MariaDB 10.1 vs. MySQL 5.7 back in 2014. This is no surprise, being that MariaDB was founded and developed under the direction of the original MySQL founder. The main advantages technically are better thread management and defragmentation of the MariaDB than MySQL databases. In addition, a larger variety of engines are available under MariaDB including NoSQL (Cassandra).

Set up PHP Repository and Certs

sudo apt-get install apt-transport-https lsb-release ca-certificates
sudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list

Update and Install PHP

Currently this is the 7.3 branch

sudo apt-get update -y
sudo apt-get install -y php7.3
sudo apt-get install -y php7.3-cli php7.3-common php7.3-curl php7.3-fpm php7.3-gd php7.3-json php7.3-mbstring php7.3-opcache php7.3-readline php7.3-xml php7.3-intl php7.3-zip
php7.3-mysql

Update and Upgrade apt

sudo apt update -y
sudo apt upgrade -y

Verify php-fpm status

systemctl status php7.3-fpm.service

stop injected data into server returns

sed -i 's/;cgi.fix_pathinfo=1/cgi.fix_pathinfo=0/g' /etc/php/7.2/fpm/php.ini
systemctl restart php7.3-fpm.service

Edit php7.3 php-fpm conf file if needed, e.g., increase upload size variables.

nano /etc/php/7.3/fpm/php-fpm.conf

Make the following changes:

cgi.fix_pathinfo = 0
...
max_execution_time = 300
...
upload_max_filesize = 32M
...
post_max_size = 32M

MariaDB - Install cert manager, key, repository

currently 10.3

sudo apt-get install -y software-properties-common dirmngr
sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8
sudo add-apt-repository 'deb [arch=amd64,i386,ppc64el] http://mirrors.dotsrc.org/mariadb/repo/10.3/debian stretch main'

Then perform update and install mariadb-server

sudo apt update -y
sudo apt-get install -y mariadb-server
sudo systemctl status mariadb

Enable auth socket

sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf

Add plugin-load-add = auth_socket.so in the [mysqld] section. Then save and restart MariaDB.

sudo systemctl restart mariadb.service

Secure the database

sudo mysql_secure_installation

PhpMyAdmin on Debian

Provided that Nginx and LetsEncrypt SSL is installed and configured. It is time to install PhpMyAdmin

sudo apt-get update
sudo apt-get install -y phpmyadmin

Add a symlink from /usr/share/phpmyadmin to /var/www/html or whatever directory for whichever website

sudo ln -s /usr/share/phpmyadmin /var/www/html

Note for security through obscurity, rename the link

sudo mv /var/www/html/phpmyadmin pma

Install and enamble mcrypt in php, and restart php-fpm

sudo apt-get install -y mcrypt
sudo phpenmod mcrypt
sudo systemctl restart php7.3-fpm

Test to see if it works

https://host.domain.tld/pma/

Limit access to /pma/ by ip address, by editing the nginx configuration

nano /etc/nginx/sites-available/default

Add the following line to the top above server:

geo $admin { default 0; 203.150.176.16 1; }

And put a nested statement under \.php as per this StackOverflow answer

location ~ \.php$ {
    location ~ (/phpmyadmin/) {          # add this
        if ($admin = 0) { return 404; }  # add this
        ## fastcgi parameters            # duplicate these lines
    }                                    # add this
    ## fastcgi parameters ##
}
Posted on

Nginx and Letsencrypt SSL on Debian

It is a good idea to get PHP and MariaDB on Debian set up before Nginx (except the PhpMyAdmin which can come after).

Related Artices in Debian Services and Applications - Debian on AWS Lightsail - OpenVPN on Debian + UFW Firewall - Nginx and Letsencrypt on Debian - PHP & MariaDB on Debian

- Grav CMS on Debian

Install Nginx

Edit the /etc/apt/sources.list to add the Nginx repostitory

nano /etc/apt/sources.list

Add the following repository (currently for Debian 9/Stretch)

deb http://nginx.org/packages/mainline/debian/ stretch nginx

Download and install the key for the repository

wget https://nginx.org/keys/nginx_signing.key
sudo apt-key add nginx_signing.key

Remove nginx-common, update apt and install nginx

sudo apt-get remove -y nginx-common
sudo apt-get update -y
sudo apt-get install -y nginx

Systemd / Nginx Race Condition

There is a known race condition, with a workaround as follows:

mkdir /etc/systemd/system/nginx.service.d
printf "[Service]\nExecStartPost=/bin/sleep 0.1\n" > /etc/systemd/system/nginx.service.d/override.conf
systemctl daemon-reload

Edit /etc/nginx/sites-available/default

Note: these edits are not comprehensive, just to get certbot working. Uncomment the following lines:

listen 443 ssl default_server;
listen [::]:443 ssl default_server;
...
location / {
...
try_files $uri $uri/ =404;
}

Where it says server_name _; change _ to an appropriate fqdn that has an appropriate A record. Save and restart the nginx:

service nginx restart

Letsencrypt Certbot

sudo apt-get update
sudo apt-get install -y python-certbot-nginx certbot -t stretch-backports

Run letsencrypt (automatic)

certbot

Test access from a browser.

HSTS Preload

Browsers have a list of servers that require https/ssl. Add sites to the list. Two things are required: 80 to 443 redirect, and an hsts header. For the redirect, add this server configuration:

server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name _;
        return 301 https://$host$request_uri;
}

For the HSTS header, this needs to be added to each server. Can simply be added after the listen 443 ssl; line:

add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;

Nginx Info

Nginx has become the standard for much of the web, for the basic standard reason it is not creaky old (though of course still lovable) Apache. However, before we get too far ahead of ourselves, let's recall exactly what we need to know about Nginx in order for it to work as well as Apache: - Installation - Configuration files - Support of SSL / LetsEncrypt - SFTP/SCP access to file system (and file rights + ownership) - Multiple virtual servers / directories - Mimetypes - Support for PHP - Threading - .htaccess and related

Nginx and Related Files and Directories

Standard or default files and directories as follows: - /etc/nginx - application directory - /etc/nginx/nginx.conf - main configuration file - /usr/share/nginx/html - default website root directory - noted as html in nginx.conf - /var/log/nginx/error.log - error log - /var/log/nginx/access.log - access log - /etc/nginx/mime.types - mime types - /etc/php.ini - php configuration file

Nginx / PHP-FPM Security Issues

There are significant issues with PHP-FPM in terms of keeping site caching partitioned when using multiple websites/virtual sites. Opcache should be turned off and individual users should be in charge of a different php-fpm process for each site. How to do this is not listed here (just yet).

Posted on

Gsuite Free Domain Alias Mailboxes

Google likes to remove functionality on free products to induce upselling. This is a common tactic in many software/SAS models. However, the cost of adopting Gsuite is very high, relative to free. Essentially a 5-10 pack of mailboxes with $5/month for the least expensive Gsuite paid option. That's $300-$600/year. What is sadly missing is a less expensive option. I don't mind paying money for valuable services, but an individual consumer who really only has family mailbox accounts, this is ridiculous pricing. As someone with multiple domains, here is how to get around this issue.

No Duplicate Mailboxes

The main problem comes when one wants to have mailboxes that have the same username, e.g., info@primary-domain.com and info@secondary-domain.com. Because added-on domains are always only aliases, only the primary domain is possible (e.g., info@primary-domain.com), and all subsequent domains with the same info@ are aliases of the underlying primary domain.

Steps to Support Duplicate Mailboxes

The work-around is as follows: - Create a unique mailbox such as secondary-domain@primary-domain.com. - After some amount of time (an hour at the most) the address info@secondary-domain.com will be added (provided info@primary-domain.com was already a primary or secondary mailbox address). - Log into secondary-domain@primary-domain.com and add info@secondary-domain.com as a second account. This will generate an email which will be sent to info@primary-domain.com. Verify access with the verification code. Set that info@secondary-domain.com as the default and configure the mailbox to always send email from that address. - Log into info@primary-domain.com and add a forwarding address of secondary-domain@primary-domain.com. This will generate a verification code emailed to secondary-domain@primary-domain.com. Verify this. - Next, create a new filter for incoming mail addressed to: info@secondary-domain.com and have it forward email to secondary-domain@primary-domain.com and also delete the email locally. The steps above will properly route and address mail so that the new mailbox will function properly using the normally disallowed duplicate username in the free version of Gsuite.

Endgame with Gsuite

Frankly I dislike Google and Gsuite. My use is only a holding action to not have to deal with email migration. The vast majority of time I no longer use Gsuite other than calendar and email, and also the use of those accounts for YouTube and Google Business Listings, and also the Analytics/Google Ads suite. Obviously there needs to be Google accounts, but they can be independent Gmail accounts rather than Gsuite accounts. At some point (in 2019), I'll migrate off and do self-hosting on mail and calendar, and therefore move YouTube, Business, Analytics over to Gmail accounts.

Posted on

Debian on AWS Lightsail

This is a setup of several items, starting with Debian 9 on Amazon AWS Lightsail. This has server basics and apt, and then follows with links to additional articles. In general, after several years of running CentOS on Linode, and then Amazon Linux AMI on EC2 and Lightsail, I find that Debian 9 is simply faster, just as secure, and at least slightly easier to use. While there are many flavors of linux, clearly two particular lineages predominate: RHEL/CentOS/AMI and Debian/Ubuntu/Mint. Either are just as valid, though of course niche requirements may make one or the other more attractive. Android and ChromeOS are even more popular, but we are dealing with server OS here. For me, Debian on the desktop via LMDE3 (Linux Mint Debian Edition) is currently a favorite.

AWS Lightsail is a decently priced VPS package. Equivalents can be found in various first and second tier cloud providers such as Digital Ocean, Vultr, Linode, and perhaps even Azure and Google Cloud, who knows? Anyone with any experience with AWS can leverage this with Lightsail, though the main web interface is a bit different.

Related Artices in Debian Services and Applications - Debian on AWS Lightsail - OpenVPN on Debian + UFW Firewall - Nginx and Letsencrypt on Debian - PHP & MariaDB on Debian

- Grav CMS on Debian

Choose Debian Distribution

On Lightsail as of late 2018 Debian 9.5 is an option. - Install PHP from special repository sources (found in the Running PHP on Debian article) - Install special packages from Backports when needed (such as certbot) - Use apt install PACKAGE -y -t stretch-backports Example:

sudo apt install -y python-certbot-nginx -t stretch-backports

Packages available from Distributions

Update Debian

sudo apt update -y
sudo apt update -y -t stretch-backports update

Upgrade Debian

Do some checks and then execute upgrade and dist-upgrade: Note: accept the locally modified files for upgrading when asked.

sudo apt upgrade -y
sudo apt upgrade -y -t stretch-backports

Note: can have system service restarts be done automatically, when asked.

Upgrade Debian Distribution

This will change from one release to the next if there is a next one for the version being run (e.g., stable).

sudo apt dist-upgrade -y

Next, run the command to reload the terminal session:

hash -r

Steps in Configuration

Server Basics Steps

  • Configure servername, ip addresses
  • Apt, Configure repositories, Update, Upgrade, Clean, etc.

Servername, IP Addresses

For private IP Addresses

ip addr show eth0 | grep inet | awk '{ print $2; }' | sed 's/\/.*$//'

For a public IP address (esp. Amazon AWS Elastic IP)

curl -4 icanhazip.com

Apt Sources List

ls -la /etc/apt

and see what is in subdirectories

Installed packages

dpkg-query -l

apt-get commands

Note, this is largely obsolete with the apt command set -- need to UPDATE this section below

apt-get clean
apt-get autoclean
apt-get dist-upgrade
apt-get clean
apt-get check
apt-get autoremove
  • autoclean deletes .deb files from local cache
  • clean deletes .deb files from distribution installation
  • autoremove removes previous, but no longer needed dependencies
  • dist-upgrade deals with dependencies, not just applications, and will add/remove/upgrade them
  • apt-get check will check for dependencies missing note: difference between apt-get remove xyz vs. apt-get purge xyz, as the first preserves configuration files (for possible later use)

Completely Remove Packages

sudo apt-get --purge remove package-name
Posted on

Niche Search

There are several white label options for search and if one considers advertising a viable business model to engage in, then consider Bing or Infospacehttp://www.infospace.com/partners/. Consider the case of Izito and related country code tlds, as well as MonsterSpace, both are pure play partner deals with nothing technical in their own right, with traffic and profitability. The best numbers come out of www.ecosia.org, which publishes their financial reports. I guess if the alternative is a search engine that doesn't provide some kind of additional value, that would be good. For sheer scale it is important to do a general purpose search engine. And then also having some level of curated content overlaid that would be black-hat proof, as well as some kind of vetting of ads?

Posted on

Xiaomi – Brand on the Rise

In my household we have been introducing Xiaomi products for the past six months. Overwhelmingly positive, but not without a hiccup here and there. In general, there is an odd mixture of: quality, design, and value. I say odd because generally those things don't go together. Great price, good looks, and works well. Not perfect, but nothing is. Xiaomi may have much bigger rivals, especially in China, but they have such a strong combination that they are able to compete with, and in some cases beat out such giants as Samsung in markets like India. Robin Li is seen as the next Steve Jobs in many ways, but one way that he is not is on price. The return of Steve Jobs to Apple was a corporate success story of huge proportions, but it certainly didn't focus on anything but the bottom line. Of course in some cases the bottom line (seen with sufficient vision) can produce quality, and even value. Really though the better comparison is an aspect of Amazon (the hardware part). While Amazon had a significant failure in producing a cell phone, their success in the Kindle, tablets and tvs, and now home devices is the real comparison for Xiaomi. Decent design, great price, reliable quality (and great customer service to bolster reliability in the case of failure).

Xiaomi Products Work Together

As with any decent ecosystem, a good product begets more product purchases, and the best products increase their value through network effects. Hence: - Xiaomi Mi Band - 900 THB on Lazada - Xiaomi Mi Box - 4K set top box - 2,700 THB on Lazada - Xiaomi Mi Redmi A4 - 3,000 THB on Ebay - Xiaomi Mi Smart Scale - 1,200 THB on Lazada - Xiaomi Mi Wireless Mouse - 500 THB on Lazada - Xiaomi Mi Earphone Piston - 300 THB on Lazada The Xiaomi Mi Fit app and integration of the Mi Band and the Mi Smart Scale is surprisingly good. The Xiaomi Mi Box Remote app is adequate, but the Mi Box remote itself works great and so the app is not needed. The Wireless Mouse is way better than bluetooth (which is generally a failure of connectivity, except where its narrow skillset is needed, such as with bluetooth speakers and smart health device connection (watch, scale, phone). The Mi Box was a great replacement for essentially all other set top boxes (when residing outside the US), including the continually crappy AppleTV, Google ChromeCast, Amazon Fire Stick, and Vero 4k.

Xiaomi Product Caution

One aspect of Xiaomi-branded products that don't work are those which need to network or communicate with the Xiaomi Home application (which has poor ratings). Though it may be a one-off experience, I could not get a Xiaomi home security camera to work. The Xiaomi Cube is very cheap, but there was no way to get it to connect to any device, and various forums point to the same problem. I'm sure it works inside China with the Chinese servers, but the external Singapore and US servers never worked, and as I'm not inside China, the Chinese servers didn't work either. The children's watch/gps looks like a great product but it is designed to only work inside China on Chinese telecom, and in fact has GMT+8 hardcoded so the timezone cannot be changed.

MIUI and MI Cloud

While the A4 hardware is excellent for the price (camera is a bit of a letdown, but sufficient for daily needs), the launcher environment and MI Cloud backup should be avoided. I've previously spent time on Android, including installing custom ROMs, so I already have a launcher I'm familiar with and suits my needs better than the MIUI interface. The MI Cloud is as exciting and functional and secure as the Apple Cloud, which means avoid and ignore as much as possible. It's definitely not needed for backup and restore when Google already does that. My guess is it is only something needed inside the Great Firewall of China.

Criticism of Design Copying

Xiaomi rightly has been criticized for copying Apple (and others) designs and if it was not located in China, it likely would have been sued into oblivion. However, there are several points to be made: - Young companies can learn from older companies by emulating (copying) them, in order to learn design in that way. This is akin to all the kinds of paintings done by Picasso long before he became a unique painter and stylist in his own right. - Apple has enabled this kind of emulation by outsourcing its production to a company which offers its services to others. What could have acted as a barrier to entry (namely, production capacity and expertise of manufacturing at scale) is something that has largely been given over to the manufacturing companies. - The lack of design protections for a non-Chinese company in China is a cost of doing business in China, wiht Chinese manufacturing. The profits reaped (due to the massive greed and focus on profits by Apple) have a downside which was known and forseen. Finally, the pricing essentially at a BOM by Xiaomi, and the learning of design and manufacturing, combined with dramatic control over inventory (they keep less inventory than any other phone device manufacturer via their flash sales), has produced a juggernaut which is more nimble than Apple, offers better prices than Apple, and is approaching the design sophistication of Apple. Once Xiaomi cellphone cameras reach parity with Apple, the Apple profit margin will dramatically at risk, as it should be.

Posted on

Dropbox Cloud Storage and Sync

Dropbox is a cloud storage and sync service, with additional editors/apps, such as Paper and Showcase. For various reasons, those additional Dropbox apps are not useful for our use cases. However, storage and sync are excellent in and of themselves, and generally superior to Google Drive which is the only real alternative.

What Dropbox gets Wrong

One thing that is maddening about Dropbox is that when renaming a folder, all files and sub-folders within the folder are re-synchronized. This can be a huge undertaking (in terms of time required, not to mention wasted bandwidth. - Dropbox Rule #1: Try not to rename folders Another thing is the slow Microsoft Online Editors for Word and Excel. These can be very tedious to use, and there is more limited functionality than found in native editors for desktop operating systems. - Dropbox Rule #2: Use a native editor on Word and Excel documents when possible. Preview for Word and Excel do not support Indic Scripts (Fonts). This means that any Thai vowel, tone mark, or silent mark will not show properly in preview (but will when editing). This is a very odd limitation, and is based on a very poor preview functionality. In contrast, nearly all other editors support Indic Scripts (South and Southeast Asia-style fonts), with the only other known exception being

Backing up Multiple Folders with Symlinks

Besides what is in the main Dropbox folder (which can be some or all of the contents), there are times when folders in other locations are needed to be included in a backup. To do this, simple create symlinks (symbolic links) from the command line. Aliases created in the Finder do not work as symlinks, so the command line is needed (or some third party app, so unnecessary). The following example points to a folder on an SDCard:

ln -s /Volumes/jm-music/iTunes-Library ~/Desktop/Dropbox/iTunes-Library-symlink
  • ln = link
  • -s = symbolic
  • /Volumes/jm-music/iTunes-Library remote link destination
  • ~/Desktop/Dropbox/iTunes-Library-symlink local link location

Dropbox vs. Google Drive for Mobile Devices

Dropbox is one of the most widely available cloud storage providers in terms of support by third-party mobile apps. While Google Drive has increased its coverage, and Microsoft lags a bit behind, Dropbox is reliably the foremost access provider for cloud storage. As well, the Dropbox app can backup images/video from mobile devices automatically.

Dropbox Desktop Sync Performance

Dropbox is a much better application for synchronization of files, in terms of stability, reliability, and resource utilization (at least on OSX). Google Drive synchronization is a nightmare of processor utilization, hangs, and error messages.

Dropbox Security Audit in Four Steps

Storage in the Cloud does not magically remove the need for security, and especially that rare creature, the security audit. From a post over at Labnol, we learned how to do a Dropbox security audit, which is important for obvious reasons. However, this requires vigilance and a repeated review, something scheduled in your calendar. Note that the user interface at Dropbox changes over time so these steps need to be updated regularly. - Last updated 18 October 2017

Step 1 - Run the Security Checkup

Run the Dropbox security checkup which reviews devices/browser, connected apps, and suggests a password change, as well as review of two-step authentication settings.

Step 2 - Review Devices and Browsers

Check the [devices and browsers which access Dropbox](https://www.dropbox.com/account/security. Anything suspicious?

Step 3 - Review Connected Apps

Review the connected apps enabled to access Dropbox. Anything suspicious?

Step 4 - Review Available Space

Check the Dropbox plan and space used/available. After all, availability is one aspect of security.

Posted on

Dropbox Paper, Markdown, Sync

Dropbox Paper is a product I really want to like. For one thing, the promise of better editor is something long unfulfilled. And taking some design cues (or perhaps merely unrelated similarities), Medium did do something nice for the blogging environment. By extending it as essentially a wysiwyg Markdown+ editor, drag and drop-friendly, with handy JavaScript handles for visual editing, this is definitely an interesting project.

Markdown as a First Class Filetype

However, there are those of us who prefer something with Markdown as a first class document filetype, which could be seamlessly synchronized alongside other files, and edited with other editors. This is what the cloud editors do, after all, provide some level of editing of desktop-class documents, collaboratively, and those same files are generally available in the same binary format (via sync or import/export). Of course by Markdown we mean much more than the anemic initial (but no less necessary) initial Markdown spec. We like Markdown Extra as a more complete specification. With Dropbox Paper there is at least one additional feature from straightforward Markdown, the inclusion of images without having to know where they physically reside. Drag in and the image appears as a part of the document. This is similar to what Github does well with its Github-flavored Markdown. Clearly there is some kind of zip/archive file format behind the scenes, which we simply don't have access to, or perhaps a nasty rats nest of pointers in a database. The thing is, Paper doesn't have a public spec or source available. In other words, Paper documents only live in Paper, the Application (akin to Google Docs). This means Paper is not a first class filetype, and therein lies the rub.

Stable File Format is Key to Offline Sync, Editor Diversity

With its Office-in-the-Cloud, Microsoft actually preserves both the file format and allows a variety of editors (which is what preserving the file format enables). This in a superior way with Word and Excel documents, essentially round-tripping edits into synced files in the desktop or the cloud. Libre Office Online, Collabora CODE, and Collabora Online are similar to the Microsoft approach, with files not changing their basic structure. Of course one would expect Microsoft to take this approach of focus on file format, since it is what helped cement their leadership in editing applications. Own the format, own the tools. Google took a different approach (for scalability reasons, surely), and the ability to edit anywhere requires offline applications and the use of a browser (which means lock-in to the Google Suite editors). Dropbox Paper is less functional than Microsoft, Libre, and Google suites, but appears to be taking a Google approach, sad.

Scalability and the Requirements for State Management

A quick read of the EtherCalc story provide excellent insight into what it takes to maintain state in a connection-less multi-user environment. Essentially a copy of the document needs to be kept updated. That generally requires the same resources as on a workstation, and was likely a strong motivation for Google building from scratch simpler non-compatible file formats for Word and Excel documents, as well as a log of all change-sets (for version tracking). The same is said for the Pydio/LibreOffice Cloud offerings, namely that they take a bit of memory to get them to work, again due to the architectural requirements of real-time server-based state management.

Limitations of Paper as a First Class Editor

Since Dropbox already uses the Microsoft Online editors for Word and Excel documents (which work as advertised), Paper is a bit of an unwanted stepchild in terms of integration. Paper (which is both a file storage and a file editor, with web and mobile app versions) doesn't live within the Dropbox folder system, but rather has its own file system. This is awkward, for navigation, to say the least. Paper folders and files do not sync.

Offline Editing with Mobile, but not Web

With the IOS or Android App, Paper files can be edited offline, akin to Google Suite, but without the ability to do offline editing with the web app. Paper files can be exported in Markdown and Word document formats, but there is no ability to import Paper files, one has to copy/paste. This is trouble if someone has a lot of Markdown files already at hand. It seems clear that this use case is fairly well ignored.

Copy/Paste and AutoCorrect in Dropbox Paper

With copy/paste, another glaring problem comes to light, which is the required transformation of single and double quotes into their fancy quotes equivalents. This is a non-starter for people working with text that needs to remain inviolable in their originally intended ASCII characters. There are obvious work-arounds for the well-known ASCII and Unicode Quotation Marks problem, and a handy visual JavaScript replacement is one that would keep the underlying text unchanged, but that is not the path which Dropbox took on this editor. Rather like Microsoft Word and Google Docs -- but unlike them without the ability to turn off the AutoCorrect options -- characters are automatically replaced, not leaving a trace of what they were formerly.

Mo' Paper, Mo' Problems

Intimated above is that what went wrong is a lack of open source of the file format that Dropbox Paper uses, which gives it severe technical limitations in terms of portability, offline editing, file synchronization, and a clear separation between interface and file format. This approach is one which Google has also embraced, and Google's moderate success in the face of such technical limitations should not be a signal of the weakness of such limitations. Rather one should look at the years and years of massive resources poured into the cloud editing project, which still cannot do proper offline file synchronization, and which has allowed Microsoft to compete effectively after a very long delay in entering the market with cloud editors.

Conclusion: Ignore Dropbox Paper

For the particular use case mentioned above, requiring file format and content integrity, file synchronization, diversity of editors, and the like, the solution is to simply ignore Dropbox Paper. The product is not for me or others with my same requirements. Fair enough. For our needs: - Dropbox for file synchronization - Stackedit version 5 for web-based editing (with access to the Dropbox file system) - Editorial for IOS is a great plain text editor supporting Markdown and Fountain (screenplay formatting), and also includes workflow scripting with python. There's a book on doing workflow with Editorial. - Atom editor Updated needs (18-Sep-2018): - Google Drive (Gsuite) + Insync for Linux - for files only, no online editors - Text - Chrome Extension, Excellent - Caret - Chrome Extension, Also Excellent - Atom editor - Native Application, Cross-Platform - Libre Office - Clunky but Functional