makefs -t ffs makes too large image

[Anthony Pankov]

Hello,

There is no much info about makefs so I ask here.

I've tried to make UFS image via `makefs` of a directory sized 500Mb and get file of 1200Mb.
Is it a bug or feature? Is it related to source files that lies on ZFS while making UFS image?

#makefs -t ffs -B little -o label=rootfs -o version=2 -o softupdates=0 image/rootfs.ufs installed

Calculated size of `image/rootfs.ufs': 1264320512 bytes, 14974 inodes
Extent size set to 32768
image/rootfs.ufs: 1205.8MB (2469376 sectors) block size 32768, fragment size 4096
using 2 cylinder groups of 866.31MB, 27722 blks, 11136 inodes

#du -hcd 1 installed/

512B installed/media
512B installed/tmp
512B installed/mnt
2,4M installed/sbin
259K installed/var
512B installed/dev
99K installed/libexec
835K installed/bin
384M installed/usr
9,7M installed/rescue
676K installed/etc
512B installed/net
5,0K installed/root
512B installed/proc
9,4M installed/lib
90M installed/boot
497M installed/
497M total

# tunefs -p image/rootfs.ufs
...
tunefs: maximum blocks per file in a cylinder group: (-e) 8192
tunefs: average file size: (-f) 16384
tunefs: average number of files in a directory: (-s) 64
tunefs: minimum percentage of free space: (-m) 8%
tunefs: space to hold for metadata blocks: (-k) 0
tunefs: optimization preference: (-o) time
tunefs: volume label: (-L) rootfs


P.S. FreeBSD version 14-STABLE

[Miroslav Lachman]

Post by Anthony Pankov
Hello,
There is no much info about makefs so I ask here.
I've tried to make UFS image via `makefs` of a directory sized 500Mb and get file of 1200Mb.
Is it a bug or feature? Is it related to source files that lies on ZFS while making UFS image?
#makefs -t ffs -B little -o label=rootfs -o version=2 -o softupdates=0 image/rootfs.ufs installed
Calculated size of `image/rootfs.ufs': 1264320512 bytes, 14974 inodes
Extent size set to 32768
image/rootfs.ufs: 1205.8MB (2469376 sectors) block size 32768, fragment size 4096
using 2 cylinder groups of 866.31MB, 27722 blks, 11136 inodes
#du -hcd 1 installed/
512B installed/media
512B installed/tmp
512B installed/mnt
2,4M installed/sbin
259K installed/var
512B installed/dev
99K installed/libexec
835K installed/bin
384M installed/usr
9,7M installed/rescue
676K installed/etc
512B installed/net
5,0K installed/root
512B installed/proc
9,4M installed/lib
90M installed/boot
497M installed/
497M total
# tunefs -p image/rootfs.ufs
...
tunefs: maximum blocks per file in a cylinder group: (-e) 8192
tunefs: average file size: (-f) 16384
tunefs: average number of files in a directory: (-s) 64
tunefs: minimum percentage of free space: (-m) 8%
tunefs: space to hold for metadata blocks: (-k) 0
tunefs: optimization preference: (-o) time
tunefs: volume label: (-L) rootfs

Is it possible that you have enabled compression on source ZFS
filesystem? Try to ad -A to you "du" command to see actual size of
files, not compressed size on filesystem.

Kind regards
Miroslav Lachman



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Anthony Pankov]

Hello Miroslav,

You are genius!

But the situation is a very frustrating. It is a default system and I did nothing to turn the compression on.
So I was absolutely sure that compression is off.

I'm sorry.
Nevertheless having compression on by default is a very weird decision and is fully unexpected for me. I've never seen a big warning about default value of this vital parameter will be inverted.

On 12 -STABLE:

# zfs get compression
NAME PROPERTY VALUE SOURCE
ps2 compression off default

On 14-STABLE

# zfs get compression
NAME PROPERTY VALUE SOURCE
tank compression on default
tank/bsdsrc compression on default

Thank you.

Post by Anthony Pankov
Hello,

Post by Anthony Pankov
There is no much info about makefs so I ask here.
I've tried to make UFS image via `makefs` of a directory sized 500Mb and get file of 1200Mb.

Is it a bug or feature? Is it related to source files that lies on ZFS while making UFS image?

Post by Anthony Pankov
#makefs -t ffs -B little -o label=rootfs -o version=2 -o softupdates=0 image/rootfs.ufs installed
Calculated size of `image/rootfs.ufs': 1264320512 bytes, 14974 inodes

Extent size set to 32768
image/rootfs.ufs: 1205.8MB (2469376 sectors) block size 32768, fragment size 4096
using 2 cylinder groups of 866.31MB, 27722 blks, 11136 inodes

Post by Anthony Pankov
#du -hcd 1 installed/
512B installed/media

512B installed/tmp
512B installed/mnt
2,4M installed/sbin
259K installed/var
512B installed/dev
99K installed/libexec
835K installed/bin
384M installed/usr
9,7M installed/rescue
676K installed/etc
512B installed/net
5,0K installed/root
512B installed/proc
9,4M installed/lib
90M installed/boot
497M installed/
497M total

Post by Anthony Pankov
# tunefs -p image/rootfs.ufs

...
tunefs: maximum blocks per file in a cylinder group: (-e) 8192
tunefs: average file size: (-f) 16384
tunefs: average number of files in a directory: (-s) 64
tunefs: minimum percentage of free space: (-m) 8%
tunefs: space to hold for metadata blocks: (-k) 0
tunefs: optimization preference: (-o) time
tunefs: volume label: (-L) rootfs

Is it possible that you have enabled compression on source ZFS filesystem? Try to ad -A to you "du" command to see actual size of files, not compressed size on filesystem.
Kind regards
Miroslav Lachman

[Miroslav Lachman]

Post by Anthony Pankov
Hello Miroslav,
You are genius!
But the situation is a very frustrating. It is a default system and I did nothing to turn the compression on.
So I was absolutely sure that compression is off.
I'm sorry.
Nevertheless having compression on by default is a very weird decision and is fully unexpected for me. I've never seen a big warning about default value of this vital parameter will be inverted.

100 people, 100 tastes :) In fact, the compression is the first thing I
turn on on each newly created pool (if it isn't the default).

Kind regards
Miroslav Lachman



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Yuri Pankov]

Post by Anthony Pankov
Hello Miroslav,
You are genius!
But the situation is a very frustrating. It is a default system and I did nothing to turn the compression on.
So I was absolutely sure that compression is off.
I'm sorry.
Nevertheless having compression on by default is a very weird decision and is fully unexpected for me. I've never seen a big warning about default value of this vital parameter will be inverted.
# zfs get compression
NAME PROPERTY VALUE SOURCE
ps2 compression off default
On 14-STABLE
# zfs get compression
NAME PROPERTY VALUE SOURCE
tank compression on default
tank/bsdsrc compression on default

It came in with the following openzfs commit and probably no one really
noticed as installer turns on compression by default, so I was going to
say it was always that way until I looked up the change :-)

commit 56fa4aa96eb3875f254e93eaef646ea20ba187f9
Author: Rich Ercolani <***@gmail.com>
Date: Thu Mar 3 13:43:38 2022 -0500

Default to ON for compression

A simple change, but so many tests break with it,
and those are the majority of this.

Reviewed-by: George Melikov <***@gmelikov.ru>
Reviewed-by: Brian Behlendorf <***@llnl.gov>
Signed-off-by: Rich Ercolani <***@gmail.com>
Closes #13078

And it looks like it's in FreeBSD starting with 14.0:

$ git branch -a --contains 56fa4aa96eb3875f254e93eaef646ea20ba187f9
* main
remotes/origin/HEAD -> origin/main
remotes/origin/main
remotes/origin/pull/956/merge
remotes/origin/releng/14.0
remotes/origin/releng/14.1
remotes/origin/stable/14


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Daniel Ebdrup Jensen]

Post by Yuri Pankov

Post by Anthony Pankov
Hello Miroslav,
You are genius!
But the situation is a very frustrating. It is a default system and I did nothing to turn the compression on.
So I was absolutely sure that compression is off.
I'm sorry.
Nevertheless having compression on by default is a very weird decision and is fully unexpected for me. I've never seen a big warning about default value of this vital parameter will be inverted.
# zfs get compression
NAME PROPERTY VALUE SOURCE
ps2 compression off default
On 14-STABLE
# zfs get compression
NAME PROPERTY VALUE SOURCE
tank compression on default
tank/bsdsrc compression on default

It came in with the following openzfs commit and probably no one really
noticed as installer turns on compression by default, so I was going to
say it was always that way until I looked up the change :-)
commit 56fa4aa96eb3875f254e93eaef646ea20ba187f9
Date: Thu Mar 3 13:43:38 2022 -0500
Default to ON for compression
A simple change, but so many tests break with it,
and those are the majority of this.
Closes #13078
$ git branch -a --contains 56fa4aa96eb3875f254e93eaef646ea20ba187f9
* main
remotes/origin/HEAD -> origin/main
remotes/origin/main
remotes/origin/pull/956/merge
remotes/origin/releng/14.0
remotes/origin/releng/14.1
remotes/origin/stable/14

Hi folks,

It's maybe also worth mentioning that compression by default makes
ZFS faster.

Allan Jude had some numbers for the default (lz4) in the
presentation he did while implementing zstd[1].

It can be a bit surprising though, but hopefully it's a good one.;)

Yours,
Daniel Ebdrup Jensen

1: https://papers.freebsd.org/2018/bsdcan/jude-zfs_zstd/

[Anthony Pankov]

Hello Daniel,

My impression that compression makes ZFS slower on VM. May be this is the only case of not so-well-optimized VM such as QEMU on Windows.

Post by Daniel Ebdrup Jensen

Post by Yuri Pankov

Post by Anthony Pankov
Hello Miroslav,
You are genius!
But the situation is a very frustrating. It is a default system and I did nothing to turn the compression on.
So I was absolutely sure that compression is off.
I'm sorry.
Nevertheless having compression on by default is a very weird decision and is fully unexpected for me. I've never seen a big warning about default value of this vital parameter will be inverted.
# zfs get compression
NAME PROPERTY VALUE SOURCE
ps2 compression off default
On 14-STABLE
# zfs get compression
NAME PROPERTY VALUE SOURCE
tank compression on default
tank/bsdsrc compression on default

It came in with the following openzfs commit and probably no one really
noticed as installer turns on compression by default, so I was going to
say it was always that way until I looked up the change :-)
commit 56fa4aa96eb3875f254e93eaef646ea20ba187f9
Date: Thu Mar 3 13:43:38 2022 -0500
Default to ON for compression
A simple change, but so many tests break with it,
and those are the majority of this.
Closes #13078
$ git branch -a --contains 56fa4aa96eb3875f254e93eaef646ea20ba187f9
* main
remotes/origin/HEAD -> origin/main
remotes/origin/main
remotes/origin/pull/956/merge
remotes/origin/releng/14.0
remotes/origin/releng/14.1
remotes/origin/stable/14

Hi folks,
It's maybe also worth mentioning that compression by default makes
ZFS faster.
Allan Jude had some numbers for the default (lz4) in the
presentation he did while implementing zstd[1].
It can be a bit surprising though, but hopefully it's a good one.;)
Yours,
Daniel Ebdrup Jensen
1: https://papers.freebsd.org/2018/bsdcan/jude-zfs_zstd/