Skip to main content

Questions tagged [filesystems]

A file system (or filesystem) is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device(s) which contain it.

Filter by
Sorted by
Tagged with
248 votes
17 answers
490k views

df in linux not showing correct free space after file removal

I have file servers which are used to store files. Files might reside there for a week, or for a year. Unfortunately, when I remove files from the server, df command doesn't reflect the freed up space....
user avatar
196 votes
6 answers
86k views

LVM dangers and caveats

I have recently started using LVM on some servers for hard drives larger than 1 TB. They're useful, expandable and quite easy to install. However, I could not find any data about the dangers and ...
Adam Matan's user avatar
  • 13.7k
191 votes
6 answers
511k views

SSL Certificate Location on UNIX/Linux

Is there any standard or convention for where SSL certificates and associated private keys should go on the UNIX/Linux filesystem?
John Topley's user avatar
  • 2,215
132 votes
10 answers
522k views

How to extend an ext4 partition and filesystem?

I have a 400GB disk with a 320GB ext4 partition. I would like to grow the ext4 partition to use the left space (80GB of free space). +--------------------------------+--------+ | ext4 ...
mimipc's user avatar
  • 2,109
129 votes
11 answers
18k views

FreeBSD: Directory called ^C (really!) - how to remove?

I made a typo: $ history 169 9:34 la /usr/local/etc/ 170 9:35 sudo mkdir ^C 171 9:36 sudo mkdir /usr/local/etc/dnsmasq.d Now I have a file that is called ^C (ctrl+C)!! When I use ls I just ...
Mausy5043's user avatar
  • 1,357
77 votes
13 answers
73k views

ZFS vs XFS

We're considering building a ~16TB storage server. At the moment, we're considering both ZFS and XFS as filesystem. What are the advantages, disadvantages? What do we have to look for? Is there a ...
Tamas Czinege's user avatar
69 votes
9 answers
133k views

How do I determine the block size of an ext3 partition on Linux?

How do I determine the block size of an ext3 partition on Linux?
mike's user avatar
  • 4,033
69 votes
10 answers
111k views

Question marks showing in ls of directory. IO errors too

Has anyone seen this before? I've got a raid 5 mounted on my server and for whatever reason it started showing this: jason@box2:/mnt/raid1/cra$ ls -alh ls: cannot access ...
user avatar
67 votes
4 answers
36k views

Drawbacks of mounting a filesystem with noatime?

Having every file be updated just when accessing them sounds like a waste. What's the catch with mounting a file system with the noatime option. What kind of applications/servers relies on the access ...
nos's user avatar
  • 2,488
65 votes
1 answer
24k views

Why are my XFS filesystems suddenly consuming more space and full of sparse files?

I've run XFS filesystems as data/growth partitions for nearly 10 years across various Linux servers. I've noticed a strange phenomenon with recent CentOS/RHEL servers running version 6.2+. Stable ...
ewwhite's user avatar
  • 199k
59 votes
3 answers
10k views

What is the difference between a soft (symbolic) link and a hard link?

I hear that you can now create soft links in Vista too. So, what is the difference between a soft (symbolic) link and a hard link on UNIX/Linux/Vista? Are there advantages of using one over the other?...
Aaron K's user avatar
  • 1,515
53 votes
3 answers
96k views

.nfsXXXX files appearing, what are those?

I have an application running (on RHEL5) that streams data onto an NFS share. Recently, I saw a lot of .nfsXXXX... (xxx being a hexadecimal number) appearing in its working directory, where the ...
nos's user avatar
  • 2,488
53 votes
7 answers
38k views

filesystem for millions of small files

Which Linux filesystem would you choose for best speed in the following scenario: a hundred million files ~2k file size on average >95% read access pretty random access high concurrency (>100 ...
bene's user avatar
  • 2,313
52 votes
14 answers
64k views

Linux - What directories should I exclude when backing up a server?

I'm backing up a Linux server and storing it on another server. I began with a simple rsync -aPh --del server.example.com:/ /mnt/backup Then someone pointed out that I shouldn't back up /proc, ...
Amandasaurus's user avatar
  • 32.5k
52 votes
4 answers
107k views

Linux/Windows/Unix/... file names: Which characters are allowed? Which are unescaped?

Which characters are allowed and which of them must be escaped on the command line in different operating systems?
java.is.for.desktop's user avatar
49 votes
7 answers
56k views

What is the best Linux filesystem for MySQL (InnoDB)?

I tried to look for benchmark on the performances of various filesystems with MySQL InnoDB but couldn't find any. My database workload is the typical web-based OLTP, about 90% read, 10% write. Random ...
Continuation's user avatar
  • 3,130
46 votes
3 answers
93k views

What is the maximum number of files a file system can contain?

Given the current structure of a directory entry on a ext4 file system on Ubuntu, what is the maximum number of files a file system can contain? What is the general method of calculating the maximum ...
user avatar
45 votes
7 answers
9k views

Removing an (apparently) infinitely recursive folder

Somehow, one of our old Server 2008 (not R2) boxes has developed a seemingly infinitely-recursing folder. This is playing havock with our backups, as the backup agent tries to recurse down into the ...
KenD's user avatar
  • 1,147
45 votes
7 answers
59k views

du vs. df difference [duplicate]

I have a fileserver where df reports 94% of / full. But according to du, much less is used: # df -h / Filesystem Size Used Avail Use% Mounted on /dev/sda3 270G 240G 17G 94%...
Andreas Kuntzagk's user avatar
44 votes
7 answers
52k views

Why can't you fsck a mounted partition?

It's well-known that you should never fsck a mounted partition. I can understand how this could easily lead to corruption if the filesystem is written to by fsck (e.g., the -a option is used), but why ...
mike's user avatar
  • 4,033
41 votes
9 answers
71k views

Need to fix file permissions in a user's home directory

Does anyone have a tool or script that will recursively correct the file permissions on a directory? On an Ubuntu Linux machine, a bunch of files were copied to a USB disk with full 777 permissions (...
mlambie's user avatar
  • 1,241
40 votes
7 answers
48k views

What happens if I delete lost+found

When you create some Linux filesystems like ext3 a 'lost+found' directory is created. According to this files will be placed there if files were damaged from some kind of system crash. What happens ...
Zoredache's user avatar
  • 132k
39 votes
3 answers
14k views

When is fsck dangerous?

Recently I've seen the root filesystem of a machine in a remote datacenter get remounted read-only, as a result of consistency issues. On reboot, this error was shown: UNEXPECTED INCONSISTENCY: RUN ...
0x5C91's user avatar
  • 493
39 votes
1 answer
7k views

Why does deleting a big file take longer?

My understanding is that running rm on a file simply unlinks it, marking the space as free in the filesystem. It should then follow that deleting one file always takes roughly the same amount of time (...
Tom Marthenal's user avatar
38 votes
5 answers
16k views

Does btrfs have an efficient way to compare snapshots?

While diffing mounted snapshots would work, it sounds like it could be horribly slow in many cases. Is there btrfs specific functionality for diffing snapshots? (I was unable to find any in the docs)
Catskul's user avatar
  • 1,969
37 votes
6 answers
39k views

Mount Docker container contents in host file system

I want to be able to inspect the contents of a Docker container (read-only). An elegant way of doing this would be to mount the container's contents in a directory. I'm talking about mounting the ...
dflemstr's user avatar
  • 583
36 votes
2 answers
7k views

Unable to write to disk but disk isn't full

I am using Ubuntu 12.04 and can't write to any file, even as root, or do any other operation that requires writing. Neither can any process that needs to write, so they're all failing. df says I've ...
felwithe's user avatar
  • 866
36 votes
2 answers
27k views

See what files are being written to (like iotop but for files, not processes)

I have a Linux server, and it's doing a lot of disk io. There is one process that is doing most of the disk writes. I found this out with iotop, which shows the processes that are doing disk I/O. Is ...
Amandasaurus's user avatar
  • 32.5k
35 votes
2 answers
22k views

Copy directory tree without empty directories?

I have the following tree # upper letters = directory # lower letters = files A |-- B |-- C |-- D |-- e <= file |-- F |-- G I need to copy this tree to another destination, ...
Dane O'Connor's user avatar
34 votes
7 answers
13k views

How beneficial are self-healing filesystems for general usage?

I have recently looked into advanced filesystems (Btrfs, ZFS) for data redundancy and availability and got interested in the additional functionality they provide, especially their "self-healing" ...
Prototype700's user avatar
34 votes
4 answers
35k views

Performance difference between ramfs and tmpfs

I need to setup an in memory storage system for around 10 GB of data, consisting of many 100 kb single files(images). There will be lots of reads and fairly periodic writes(adding new files, deleting ...
Ivan Kovacevic's user avatar
33 votes
9 answers
51k views

Best way to compare (diff) a full directory structure?

What's the best way to compare directory structures? I have a backup utility which uses rsync. I want to tell the exact differences (in terms of file sizes and last-changed dates) between the source ...
Adam Matan's user avatar
  • 13.7k
33 votes
4 answers
76k views

Is there such a thing as a 'quick' format for ext4?

In Windows, it's very fast to format with NTFS. I have a low powered Linux machine, with little RAM. Formatting a 2TB volume to ext4 takes a long long time. Is there anything I can do to speed up the ...
stuck's user avatar
  • 697
33 votes
8 answers
48k views

How to workaround the NTFS Move/Copy design flaw?

As anyone that has dealt with file server permissions is aware, NTFS has an interesting design feature/flaw known as the Move/Copy problem. As explained in this MS KB article, the permissions for a ...
David Archer's user avatar
32 votes
6 answers
5k views

What are .DS_Store files and why does OSX leave them on Windows shares?

Whenever I access windows shares from OSX 10.5 it leaves .DS_Store files on the remote filesystem. What are they used for, and are they necessary, and can they be prevented from being created?
duckworth's user avatar
  • 700
32 votes
5 answers
13k views

What are the performance implications for millions of files in a modern file system?

Let's say we're using ext4 (with dir_index enabled) to host around 3M files (with an average of 750KB size) and we need to decide what folder scheme we're going to use. In the first solution, we ...
Leandro Moreira's user avatar
31 votes
5 answers
8k views

Turning off atime on a filesystem

I am setting up a mongoDB replica set and one of the first things I am suppose to do is turn off atime on the file system. After researching this a bit, I am not opposed to doing this, but I have to ...
QWade's user avatar
  • 927
31 votes
11 answers
21k views

Distributed Storage Filesystem - Which one/Is there a ready to use product?

With Hadoop and CouchDB all over in Blogs and related news what's a distributed-fault-tolerant storage (engine) that actually works. CouchDB doesn't actually have any distribution features built-in, ...
31 votes
3 answers
18k views

The XFS filesystem is broken in RHEL/CentOS 6.x - What can I do about it?

Recent versions of RHEL/CentOS (EL6) brought some interesting changes to the XFS filesystem I've depended on heavily for over a decade. I spent part of last summer chasing down an XFS sparse file ...
ewwhite's user avatar
  • 199k
30 votes
2 answers
59k views

Transparent compression filesystem in conjunction with ext4

I am trying to test a project that needs compressed storage with use of the ext4 file system since the application I use relies on ext4 features. Are there any production/stable solutions out there ...
user235918's user avatar
29 votes
6 answers
19k views

Preferred format of file names which include a timestamp

As we all know "unix" can have anything in a file except '/' and '\0', sysadmins however tend to have a much smaller preference, mainly due to nothing liking spaces as input ... and a bunch of things ...
James Antill's user avatar
27 votes
6 answers
35k views

What's the maxium number of files a Unix folder can hold?

What's the maxium number of files a Unix folder can hold? I think it will be the same as the number of files.
user avatar
26 votes
5 answers
61k views

How do I identify the file system of a drive in Mac OS?

I have an external drive hooked up to my Mac, and I'm trying to determine things like, e.g., is this HFS or FAT, is it 32-bit or 64-bit, etc. It seems like there should be some trivial command that ...
David Moles's user avatar
26 votes
12 answers
8k views

Storing and backing up 10 million files on Linux

I run a website where about 10 million files (book covers) are stored in 3 levels of subdirectories, ranging [0-f]: 0/0/0/ 0/0/1/ ... f/f/f/ This leads to around 2400 files per directory, which is ...
BenMorel's user avatar
  • 4,607
26 votes
2 answers
8k views

ZFS Data Loss Scenarios

I'm looking toward building a largish ZFS Pool (150TB+), and I'd like to hear people experiences about data loss scenarios due to failed hardware, in particular, distinguishing between instances where ...
Cyclone's user avatar
  • 567
25 votes
2 answers
7k views

In Linux, why isn't superuser root's folder under /home?

You can create a user that has privileges like root, and it's home directory will fall under /home/username. Why does root get its own folder at the top level of the file system? Is this just ...
tacos_tacos_tacos's user avatar
25 votes
7 answers
5k views

Do I need to defrag unix filesystems?

I know Windows file systems are very sensitive to fragmentation and Unix such as UFS, ZFS, ext3, or ext4 are not as sensitive. But, is there any reason to defragment these filesystems at all? If so, ...
paulgreg's user avatar
  • 4,174
25 votes
9 answers
23k views

Maximum number of files in one ext3 directory while still getting acceptable performance?

I have an application writing to an ext3 directory which over time has grown to roughly three million files. Needless to say, reading the file listing of this directory is unbearably slow. I don't ...
knorv's user avatar
  • 1,819
25 votes
2 answers
19k views

How to track all files a process opens in its lifetime

I am aware of using lsof for checking the files currently accessed by a process. Does there exist a way to see all files that an application opens in its lifetime?
Peter Smit's user avatar
  • 1,689
24 votes
5 answers
21k views

last time a filesystem was checked?

What command can tell me the last time that a filesystem was fsck'd? A date would be nice, but I'd settle for the mount count since the last fsck. I've looked all around for this in fsck*, lsattr, ...
Andrew Schulman's user avatar

1
2 3 4 5
40