NextCloud: Install: Heroku

From Gratiel's Wiki
Revision as of 15:27, 31 July 2019 by Ewanchic (talk | contribs) (→‎Install NextCloud - Phase 1)

Preface

  • Ephemeral Drive
    • Locked Code
    • S3
  • Databases
    • PostgreSQL

Prerequisites

You are going to need the following Services and Software before getting started with your NextCloud installation on Heroku. 

Be sure you have accounts setup for the following Services listed below. 
Also, be sure you have installed the following Software listed below.

Services

Software

sudo snap install heroku --classic

Setup Heroku

Login to Heroku CLI (Ubuntu)

heroku login

Create App

  • This needs to be created after git. IT will automatically create the remote links needed in Git
heroku create infinite-lowlands-38883

Heroku will return a sample app and location. Use this for the rest of the install. 

Creating app... done, ⬢ infinite-lowlands-38883
https://infinite-lowlands-38883.herokuapp.com/ | https://git.heroku.com/infinite-lowlands-38883.git

Destroy App

heroku apps:destroy infinite-lowlands-38883 --confirm infinite-lowlands-38883

Destroying ⬢ infinite-lowlands-38883  (including all add-ons)... done

Heroku Add-ons

  • Heroku Postgres

Install NextCloud - Phase 1

Download & Extract NextCloud

Download the version of nextcloud 

wget https://download.nextcloud.com/server/releases/nextcloud-15.0.8.zip

Unzip nextcloud-15.0.8.zip to it's own folder 

unzip nextcloud-15.0.8.zip

Create a new folder `nextcloud-15.0.8` 

mkdir nextcloud-15.0.8

Move folder `nextcloud` into `nextcloud-15.0.8` and label it `server` 

mv nextcloud nextcloud-15.0.8/server

Initialize Git

cd nextcloud-15.0.8
git init
git config user.name "Eric Wanchic"
git config user.email "ewanchic@elifestream.com"

cat > .gitignore

Copy & Paste the following to the file: 

/vendor/
.env

Ctrl+D 

git add .
git commit -m "Initial NextCloud 15.0.8"

Setup Composer

cat > composer.json

Copy & Paste the following to the file: 

{
  "require" : {
    "php": "7.2.19",
    "ext-mbstring": "*",
    "ext-gd": "*",
    "ext-intl" : "*",
    "ext-imagick": "*",
    "ext-memcached": "*",
    "ext-pcntl": "*",
    "ext-apcu": "*"
  },
  "require-dev": {
    "php": "7.2.19",
    "ext-mbstring": "*",
    "ext-gd": "*",
    "ext-intl" : "*",
    "ext-imagick": "*",
    "ext-memcached": "*",
    "ext-pcntl": "*",
    "ext-apcu": "*"
  }
}

Ctrl+D to exit 

composer update

Commit code to repo 

git add .
git commit -m "Adding composer files."

Setup PHP FPM file

cat > fpm_custom.conf

Copy & Paste the following to the file: 

php_value[upload_max_filesize] = 300M
php_value[post_max_size] = 200M
php_value[memory_limit] = 128M
php_value[mbstring.func_overload] = 0
php_value[default_charset] = 'UTF-8'
php_value[output_buffering] = 0

php_value[apc.enable_cli] = 1

Ctrl+D to exit

Commit code to repo 

git add .
git commit -m "Adding custom php fpm."


Setup Procfile

cat > Procfile

Copy & Paste the following to the file: 

web: vendor/bin/heroku-php-apache2 -F fpm_custom.conf server/

Ctrl+D to exit

You might be tempted to use nginx instead of apache. In this instance, stay with apache.

Commit code to repo 

git add .
git commit -m "Adding Procfile."

Install PostgreSQL Add on

heroku addons:create heroku-postgresql:hobby-dev --app infinite-lowlands-38883

DB is created. You'll have to log in to get further details. 

Creating heroku-postgresql:hobby-dev on ⬢ infinite-lowlands-38883... free
Database has been created and is available
 ! This database is empty. If upgrading, you can transfer
 ! data from another database with pg:copy
Created postgresql-crystalline-20674 as DATABASE_URL
Use heroku addons:docs heroku-postgresql to view documentation

Install NextCloud - Phase 2

Pushing Code to Heroku

git push heroku master

Do not setup the app just yet.

SSH into App

Need to SSH into Heroku after app starts. 

heroku ps:exec --app infinite-lowlands-38883

Running this command for the first time requires a dyno restart.
Do you want to continue? [y/n]: y
Initializing feature... done
Restarting dynos... done
Waiting for web.1 to start... done
Establishing credentials... done
Connecting to web.1 on ⬢ infinite-lowlands-38883... 
~ $

Get the current Config

cd server/config
cat config.php

<?php
$CONFIG = array (
  'instanceid' => 'oczm2c0yitlu',
);

Create a config

cat > config.php
<?php
$CONFIG = array (
  'instanceid' => 'oczm2c0yitlu',
);

Ctrl+D 

git add .
git commit -m "Initializing config.php"

Install NextCloud - Phase 3

Run the Setup

  1. Running the setup, Add username, password, and PostgreSQL information.
  2. Setup will Fail. Error: The username is already being used

Get new config file

  1. Log into Heroku again
heroku ps:exec --app infinite-lowlands-38883
Establishing credentials... done
Connecting to web.1 on ⬢ infinite-lowlands-38883...
  1. Output config file:
<?php
$CONFIG = array (
  'instanceid' => 'oci1sj5gwidj',
  'passwordsalt' => '2YHY53pJ19z1sQH8VLlyxQnJ+G7X/n',
  'secret' => 'UWNfUzFSNDmqTLNwhtsY3FEvg8PfDND8mUfcKmBV78UQweAS',
  'trusted_domains' => 

  array (
    0 => 'infinite-lowlands-38883.herokuapp.com',
  ),
  'datadirectory' => '/app/server/data',
  'dbtype' => 'pgsql',
  'version' => '15.0.8.1',
  'overwrite.cli.url' => 'http://infinite-lowlands-38883.herokuapp.com',
  'dbname' => 'd8saj3o99k4kf6',
  'dbhost' => 'ec2-54-225-72-238.compute-1.amazonaws.com',
  'dbport' => ,
  'dbtableprefix' => 'oc_',
  'dbuser' => 'tvsinoicrvngaq',
  'dbpassword' => 'cb6e7c80e137f4dd06011f1319e1537e81066c66ec9a527fc75980b87d71de82',
);

This step not only establishes the database and site credentials (which we could have added manually), but it sets up the: username & password, password salt, and secretkey which all are encoded into the database.

We are going to need to save these values and enter them into Heroku's Config Vars.

Now we need to modify the config.php to accept environment variables: 

<?php
$CONFIG = array (
  'instanceid' => 'oci1sj5gwidj',
  'passwordsalt' => getenv('PASSWORD_SALT'),
  'secret' => getenv('SECRET'),
  'trusted_domains' => 

  array (
    0 => 'infinite-lowlands-38883.herokuapp.com',
  ),
  'datadirectory' => getenv('DATA_DIRECTORY'),
  'dbtype' => 'pgsql',
  'version' => '15.0.8.1',
  'overwrite.cli.url' => 'http://infinite-lowlands-38883.herokuapp.com',
  'dbname' => getenv('DB_NAME'),
  'dbhost' => getenv('DB_HOST'),
  'dbport' => ,
  'dbtableprefix' => 'oc_',
  'dbuser' => getenv('DB_USER'),
  'dbpassword' => getenv('DB_PASSWORD'),
);


Go ahead and update the config.php on your local git repo to match this. Save and commit. 

git add .
git commit -m "Updated config.php. Includes DB credentials, trusted site credentials, passwordsalt & secretkey. All important values are being replaced by Environment Variables."

Install NextCloud - Phase 4 (S3 Bucket)

  • Setup S3 Logging for two buckets
    • Actual S3 Bucket
    • The Replicated S3 Bucket
  • Setup Replication for an S3 bucket
    • Setup Versioning
    • Link the S3 Bucket to it's log
    • Block All Public Access
  • Create the S3 Bucket
    • Setup Versioning
    • Link the S3 Bucket to it's log
    • Link the S3 Bucket to replication
    • Block All Public Access
  1. Configure config.php file
<?php
$CONFIG = array (
  'instanceid' => 'oci1sj5gwidj',
  'objectstore' =>
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' =>
    array (
      'bucket' => getenv('S3_BUCKET'),
      'autocreate' => true,
      'key' => getenv('S3_KEY'),
      'secret' => getenv('S3_SECRET'),
      'use_ssl' => true,
      'region' => getenv('S3_REGION'),
      'use_path_style' => false,
    ),
  ),
  'passwordsalt' => getenv('PASSWORD_SALT'),
  'secret' => getenv('SECRET'),
  'trusted_domains' => 

  array (
    0 => 'infinite-lowlands-38883.herokuapp.com',
  ),
  'datadirectory' => getenv('DATA_DIRECTORY'),
  'dbtype' => 'pgsql',
  'version' => '15.0.8.1',
  'overwrite.cli.url' => 'http://infinite-lowlands-38883.herokuapp.com',
  'dbname' => getenv('DB_NAME'),
  'dbhost' => getenv('DB_HOST'),
  'dbport' => ,
  'dbtableprefix' => 'oc_',
  'dbuser' => getenv('DB_USER'),
  'dbpassword' => getenv('DB_PASSWORD'),
);

Go ahead and update the config.php on your local git repo to match this. Save and commit. 

git add .
git commit -m "Updated config.php. Includes S3 credentials, important values are being replaced by Environment Variables."


Need to apply the patch. 

git add .
git commit -m "Apply S3 Patch".

Install NextCloud - Phase 5 (Tell NextCloud that it is done installing)

Obtain config #2

git push heroku master
  • Check everything, code should be clean.
  • Reset the Database
  • Redo the install
  • Check that the S3 bucket was filled with oids.
  • look at the new config.php File

Obtain new Values

  • If there is a new config file, then redo the environment vars with:
    • Secret
    • Password-Salt

Set Install status to True

Add the new item to tell the config.php that the install is completed. 

'installed' => true,

git commit -a -m "Telling NextCloud that its installation process is finished."

Configure Log Files

Add some additional information for logging purposes. 

 'log_type' => 'syslog',
 'logfile' => ,
 'loglevel' => '1',
 'logdateformat' => 'F d, Y H:i:s',

git commit -a -m "Adding some additional logging parameters for nextcloud. Sending log info to syslog."

Add File Caching

Add the new item to tell the config.php that the install is completed. 

'memcache.local' => '\\OC\\Memcache\\APCu',

git commit -a -m "Add File Caching with APCu."

Add ghost files

mkdir -p server/data
cd server/data
touch .ocdata index.html nextcloud.log
cd ../..

git add .
git commit -m "Adding ghost files needed in ./data."

Recommit and Test

git push heroku master

Check that the login works.

Install NextCloud - Phase 6 (Install Apps)

Log into bash app 

heroku ps:exec --app infinite-lowlands-38883

Start a Git Repo 

git init
git add .
git config user.name "Eric Wanchic"
git config user.email "ewanchic@elifestream.com"
git commit -m "Init App"

Impersonate

  • Install the Impersonate App
    • Apps -> Tools -> Impersonate
    • Download and Install
    • server/apps/impersonate/

See what files have been installed

Auditing / Logging

  • Enable Auditing / Logging

Install NextCloud - Phase 7 (Security)

TOTP

git add .
git commit -m "Two-Factor TOTP App"
git push heroku master
  • Enable TOTP
  • User Profile
    • Personal-> Security
    • Enable TOTP
    • Authenticate Code
  • Add TOTP config Items
'twofactor_enforced' => 'true',
'twofactor_enforced_groups' =>
array (
),
'twofactor_enforced_excluded_groups' =>
array (
),

Users

  • Remove All Demo Files
  • Set User file Quota from Unlimited to 0 B

Install NextCloud - Phase 8 (S3 Storage)

  • Enable App 'External storage support'
  • Refresh the Page.
  • Settings -> Administration -> External storages

Add a Bucket

  • Add a New Bucket
    • AmazonS3
    • Bucket Name
    • Region
    • Enable SSL
    • Access Key
    • Access Secret
    • Set User
    • Enable Previews
    • Enable Sharing
    • Check for Changes -> Never
    • Click the Check to Save

Test the Bucket

  • Drag a file to Save.
  • Check on S3 Bucket, original files
  • Check on S3 Bucket, clonetrail files

Install NextCloud - Phase 9 (Patching NextCloud)

Errors

  • Some files have not passed the integrity check. Further information on how to resolve this issue can be found in the documentation. (List of invalid files… / Rescan…)

Click the list of Invalid files. My example: 

Technical information
=====================
The following list covers which files have failed the integrity check. Please read
the previous linked documentation to learn more about the errors and how to fix
them.

Results
=======
- gallery
	- FILE_MISSING
		- .gitmodules
- files_external
	- INVALID_HASH
		- lib/Lib/Storage/AmazonS3.php

Raw output
==========
Array
(
    [gallery] => Array
        (
            [FILE_MISSING] => Array
                (
                    [.gitmodules] => Array
                        (
                            [expected] => 2ff42782d6cd17d3d172f3c367c7c9b14737242bd2a07ac08fd38ec00e2ec0e5abd12fd6d7dc822c226ffa2500b610c4468cbd96793bc8a3fe54744a1f7259e9
                            [current] => 
                        )

                )

        )

    [files_external] => Array
        (
            [INVALID_HASH] => Array
                (
                    [lib/Lib/Storage/AmazonS3.php] => Array
                        (
                            [expected] => 730faa8c9d4212f6940ae68c3cbf95297d7875ba753e5ea7d6103481e3eb6327058ba4efd5134a0a21e0f1727d2d88b6acd62ea20a50f6ecdb56ca5fe14a13c1
                            [current] => 4fc677f9e6586f856a58b48f31bb1fb8022383990875379ef68dc502967440cfefd13a611e7993fab2294108f6a6ab35033f708902d7f9fd344e01f10e729af2
                        )

                )

        )

)

HTTPS

  • Accessing site insecurely via HTTP. You are strongly adviced to set up your server to require HTTPS instead, as described in the security tips ↗.

WebDAVs

  • Your web server is not properly set up to resolve "/.well-known/caldav". Further information can be found in the documentation.
  • Your web server is not properly set up to resolve "/.well-known/carddav". Further information can be found in the documentation.

Database Fix

  • Some columns in the database are missing a conversion to big int. Due to the fact that changing column types on big tables could take some time they were not changed automatically. By running 'occ db:convert-filecache-bigint' those pending changes could be applied manually. This operation needs to be made while the instance is offline. For further details read the documentation page about this.
    • filecache.mtime
    • filecache.storage_mtime

Log into bash app 

heroku ps:exec --app infinite-lowlands-38883

Navigate to code and run DB fix: 

~ $ cd server/

Temporarily replace the config/config.php with al the environmental variable replaced. 

<?php
$CONFIG = array (
  'instanceid' => 'oczm2c0yitlu',
  'objectstore' =>
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' =>
    array (
      'bucket' => $_SERVER['S3_BUCKET'],
      'autocreate' => true,
      'key' => $_SERVER['S3_KEY'],
      'secret' => $_SERVER['S3_SECRET'],
      'use_ssl' => true,
      'region' => $_SERVER['S3_REGION'],
      'use_path_style' => false,
    ),
  ),
  'memcache.local' => '\\OC\\Memcache\\APCu',  

  'passwordsalt' => $_SERVER['PASSWORD_SALT'],
  'secret' => $_SERVER['SECRET'],
  'trusted_domains' => 
  array (
    0 => 'infinite-lowlands-38883.herokuapp.com',
  ),
  'datadirectory' => $_SERVER['DATA_DIRECTORY'],
  'dbtype' => 'pgsql',
  'version' => '15.0.8.1',
  'overwrite.cli.url' => 'http://infinite-lowlands-38883.herokuapp.com',
  'dbname' => $_SERVER['DB_NAME'],
  'dbhost' => $_SERVER['DB_HOST'],
  'dbport' => '5432',
  'dbtableprefix' => 'oc_',
  'dbuser' => $_SERVER['DB_USER'],
  'dbpassword' => $_SERVER['DB_PASSWORD'],
  'installed' => true,  
  'log_type' => 'syslog',
  'logfile' => ,
  'loglevel' => '1',
  'logdateformat' => 'F d, Y H:i:s',  
  'twofactor_enforced' => 'true',
  'twofactor_enforced_groups' =>
  array (
  ),
  'twofactor_enforced_excluded_groups' =>
  array (
  ),  
);

Then it will run: 

~/server $ php occ db:convert-filecache-bigint
The current PHP memory limit is below the recommended value of 512MB.
Following columns will be updated:

* filecache.mtime
* filecache.storage_mtime 

This can take up to hours, depending on the number of files in your instance!
Continue with the conversion (y/n)? [n] y
~/server $

Restart Heroku system. 

heroku restart --app infinite-lowlands-38883

Install NextCloud - Phase 10 (WebDAV)

Patches

Heroku

More than one MPM loaded.

Heroku - AH00534: apache2: Configuration error: More than one MPM loaded.

To fix this, you need to execute this patch to heroku: 

heroku labs:enable --app infinite-lowlands-38883 runtime-new-layer-extract
Enabling runtime-new-layer-extract for infinite-lowlands-38883... done

NextCloud

S3 Issues

Commands

Heroku

Restart 

heroku ps:restart --app infinite-lowlands-38883
Restarting dynos on ⬢ infinite-lowlands-38883... done

NextCloud

Backup

Restore

Upgrading

References

Return

Retrieved from "https://wiki.gratiel.com/index.php?title=NextCloud:_Install:_Heroku&oldid=91"